E(n) Equivariant Topological Neural Networks

We’d like to thank the Reviewer for their feedback, as well as the raised concerns and questions. Also, we’d like to thank them for identifying as strengths of the paper both the framework and the theoretical contributions, the empirical results, as well as the applications and the novel dataset. Next, we address each raised weakness in detail, and also point them out as answers to their questions.

Weaknesses.

[W1] Novelty

On the one hand, it is true that ETNNs are scalarization-based architectures, as EGNNs [1], on higher-order combinatorial domains, as EMPSNs [2]. As such, they inevitably resound with each other. However, we kindly disagree with the reviewer’s comment since ETNNs are a formal and more expressive generalization of both EMPSNs and EGNNs. For this reason, ETNNs unlock several features (e.g. arbitrary modeling of heterogeneous hierarchical higher-order interactions, tunable expressive power, and general learnable geometric invariants) that the constrained graph or simplicial structures of EGNNs and EMPSNs cannot accommodate (see Appendix C and F). As such, our framework can be used to design arbitrary E(n) equivariant TDL models. No other framework has the same power.

Moreover, although we believe that the generality of our framework is fundamental, we also recognized the necessity for applications whose exhaustive modeling (in a combinatorial/topological sense) is possible only via combinatorial complexes, and we introduced MolecularCCs and GeospatialCCs to tackle this problem (no other complex nor graph can model molecular and multi-resolution irregular geospatial data as straightforwardly as CCs). As a consequence, we achieved or matched SotA results among the Equivariant TDL models with a huge reduction in computational complexity. As a byproduct, as reviewer NQcw noticed too, our air pollution downscaling task represents a novel benchmark for the TDL community, addressing a need highlighted in the recent position paper [3].

Finally, we believe that the expressivity analysis is novel. In particular, our approach is, to the best of our knowledge, currently the most exhaustive one for scalarization-based equivariant TDL models, as it could be applied to analyze the (geometric WL) expressivity of any scalarization-based equivariant TDL model without the need for domain-specific coloring procedures.

[W2] Expressivity on CTSs

Our choice of studying expressivity in terms of distinguishing geometric graphs (empirically, we have the counterexamples experiment too, not only the $k$-hop distinct graphs experiment) is due to 3 main reasons:

(i) Equivariant TDL is an extremely recent (sub-)field, thus it made sense to us to first show that ETNNs are more expressive than scalarization-based graph models (i.e., EGNNs). This is something already new, as neither [2] nor [4] show any expressivity result.

(ii) To achieve (i), we wanted to rely on strong and recognized theoretical tools, and the Geometric WL (and related tasks, e.g., $k$-hop distinct graphs and counterexamples) looked like a natural choice.

(iii) The approach we adopt to port the Geometric WL machinery in our setting is itself completely novel and represents a versatile and general approach for studying the (geometric WL) expressivity of any scalarization-based equivariant TDL model that can be derived from the ETNN framework.

This said, we undoubtedly agree with the reviewer's comment. Having a higher-order geometric WL (and related tasks, e.g., $k$-hop distinct complexes) to distinguish among geometric realizations of CTSs is one of the most interesting directions. However, adapting the entire topological-/k-WL machinery to the geometric setting is a highly non-trivial task, and situating it within the context of existing literature adds further complexity. We believe that addressing these challenges fully would necessitate a separate dedicated paper.

[W3] Statement of Proposition 2

The reviewer is correct that we need to refine the statement to make it clearer. In the revised manuscript, we clearly state the condition of improved expressivity induced by Proposition 2 in the updated Corollary 1. In particular, an ETNN is strictly more powerful than an EGNN in distinguishing $k$-hop distinct graphs if the skeleton-preserving lifting is such that the number of ETNN layers required to have at least one cell in $X_{G_1}$/$X_{G_2}$ whose receptive field is the whole set of nodes in $G_1$/$G_2$ is smaller than the number of EGNN layers required to have at least one node in $G_1$/$G_2$ whose receptive field is the whole set of nodes in $G_1$/$G_2$.

Continuing response in the thread (References are included in the last part of the thread)

[W4] ETNN vs EMPSN expressivity

As a consequence of what we wrote in our reply to W1, a comparison is not required, because CCs generalize Cellular/Simplicial Complexes (SCs) and our approach based on the novel notion of Geometric Augmented Hasse Graph can thus be directly applied to SCs. As we carefully explained in the paper, an expressivity comparison (in the geometric WL sense) is fair only when the lift is skeleton-preserving, i.e. when the underlying graph connectivity is preserved; in [2], they do not lift the graphs (and they do not provide any expressivity analysis), but rather they replace the connectivity with a Vietori-Rips SC. Although one can indeed lift a graph into a simplicial complex preserving the graph connectivity by taking triangles or cliques as higher-order simplices, the expressivity would be characterized exactly as for Proposition 2 (and as in [W3] to compare with GNNs).

[W5] Statistical Significance on QM9

At the beginning of Appendix J, we exhaustively explain why we don’t run the model on multiple splits (each one of the baseline models uses different splits) and, in general, our meticulous care for reproducibility and fairness. Practical examples of this are also the fact that, despite its irreproducibility, we cared a lot about a fair comparison with EMPSN, or with an EGNN having our same parameter budget. Overall, for the QM9 benchmark, we followed the established experimentation setup of this specific dataset, following prior works, in which authors reported their best performance results and we were completely transparent about it (see Appendix J and I). This can be resonated by the large size of the molecular dataset, which makes multiple iterations of experiments challenging. In this sense, ETNN sets an unprecedented standard in the TDL community regarding exhaustiveness and transparency in the presentation of the results.

[W6] Statistical Significance on Air Pollution Downscaling

We originally did not report variances since our experiments are averaged over 3 seeds, which makes variance calculations unreliable. We are now working on running with additional seeds and will report the results when the experiments are complete.

Questions.

[Q1] K-hop distinct graphs

Please see [W2].

[Q2] ETNN vs EMPSN expressivity

Please see [W4].

[Q3] Statement of Proposition 2

Please see [W3]

[Q4] Statistical Significance

Please see [W5]-[W6].

References

[1] Víctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. "E(n) Equivariant Graph Neural Networks." In International Conference on Machine Learning, pages 9323–9332. PMLR, 2021.

[2] Floor Eijkelboom, Rob Hesselink, and Erik J Bekkers. "E(n) Equivariant Message Passing Simplicial Networks." In International Conference on Machine Learning, pages 9071–9081. PMLR, 2023.

[3] Papamarkou, Theodore, et al. "Position: Topological Deep Learning is the New Frontier for Relational Learning." Forty-first International Conference on Machine Learning, 2024.

[4] Liu, Cong, et al. "Clifford group equivariant simplicial message passing networks." ICLR 2024.

I thank the reviewers for thorough responses, which have strengthened my understanding of this work. The paper presents good contributions that merit publication, and I'd like to address the clarifications while explaining why I maintain my current score.

[W1] The authors' response clarifies the key differences between ETNNs and EMPSNs, particularly regarding neighborhood functions and geometric invariants. While I now have a clearer technical understanding of these differences, I view them as adaptations for handling CCs rather than fundamental architectural innovations. As I noted in my review, this contribution is still valuable – I simply maintain this as a non-detrimental weakness.

[W2] The authors' clarification about Remark 6 and the example of undistinguishable geometric CCs is helpful, and I think the paper would benefit from highlighting it in the main text. However, two points remain: (a) while instructive, this specific example doesn't easily generalize, and it’s unclear how we can establish a more comprehensive expressivity hierarchy, and (b) it demonstrates one direction (indistinguishable geometric CCs) without fully characterizing which geometric CC classes can be distinguished. I again want to stress that this is a non-detrimental weakness and that, as I pointed out in my review, the paper's theoretical contributions are valuable.

[W3] I understand the authors' argument re: the expressivity comparison (for $k$-chains and counterexamples) through Tables 3 and 4 + the fact that EMPSN with skeleton-preserving lifting would be limited to EGNN expressivity in those cases. This is a subtle argument that mixes empirical and theoretical observations. A formal theoretical characterization would be valuable in future work.

[W4] The additional seeds for the geospatial experiment strengthen the empirical validation and are a good precedent for the benchmark.

In conclusion, I maintain my positive assessment of the paper. The paper makes valuable contributions to equivariant topological deep learning, particularly through its practical innovations and empirical results. While the noted limitations prevent me from increasing my score, they don't detract from the paper's value and publishability.

We would like to thank the Reviewer for both pointing out the paper’s strengths, as well as their raised concerns regarding novelty, expressivity and the comparison with the CCMPNs. Next, we address these points.

Weaknesses.

[W1] Novelty

On the one hand, it is true that ETNNs are scalarization-based architectures, as EGNNs [4], on higher-order combinatorial domains, as EMPSNs [2]. As such, they inevitably resound with each other.

However, we kindly disagree with the reviewer’s comment since ETNNs are a formal and more expressive generalization of both EMPSNs and EGNNs. For this reason, ETNNs unlock several features (e.g. arbitrary modeling of heterogeneous hierarchical higher-order interactions, tunable expressive power, and general learnable geometric invariants) that the constrained graph or simplicial structures of EGNNs and EMPSNs cannot accommodate (see Appendix C and F). As such, our framework can be used to design arbitrary E(n) equivariant TDL models. No other framework has the same power.

Moreover, although we believe that the generality of our framework is fundamental, we also recognized the necessity for applications whose exhaustive modeling (in a combinatorial/topological sense) is possible only via combinatorial complexes, and we introduced MolecularCCs and GeospatialCCs to tackle this problem (no other complex nor graph can model molecular and multi-resolution irregular geospatial data as straightforwardly as CCs). As a consequence, we achieved SotA results or matched the existing SotA results among the Equivariant TDL models with a huge reduction in computational complexity. As a byproduct, as reviewer NQcw noticed too, our air pollution downscaling task represents a novel benchmark for the TDL community, which can be proven valuable to its needs, as noticed in the position paper [5].

Finally, we believe that the expressivity analysis is novel. In particular, our approach is, to the best of our knowledge, currently the most exhaustive one for scalarization-based equivariant TDL models, as it could be applied to analyze the (geometric WL) expressivity of any scalarization-based equivariant TDL model without the need for domain-specific coloring procedures.

[W2] Expressivity and Geometric Invariance

Our expressivity analysis in Appendix F assumes that the employed geometric invariant is the sum of pairwise distances. We believe that this is already the simplest and most computationally efficient geometric invariant. Moreover, as discussed in Remark 5, the assumptions of Proposition 1 are required for a clean theoretical treatment, but we empirically observed that different aggregations, as well as different message functions and geometric invariants, do not affect the expressiveness. This makes sense, as the assumptions in Proposition 1 lead to the simplest possible architectural setting for an ETNN. About what geometric invariant functions graph models are unable to compute/learn, we are not sure we understood what the reviewer meant. The geometric invariants take as input all the geometric features of the nodes belonging to a certain cell. As such, a graph cannot compute/learn anything that takes as input more than two node geometric features (i.e., what is induced by the presence of an edge), as it cannot jointly access higher-order information (i.e., geometric features of more than two nodes).

[W3] ETNN vs EGNN expressivity

The reviewer is correct that we need to refine the statement to make it clearer. In the revised manuscript, we clearly stated the condition of improved expressivity induced by Proposition 2 in the new Corollary 1. In particular, an ETNN is strictly more powerful than an EGNN in distinguishing $k$-hop distinct graphs if the skeleton-preserving lifting is such that the number of ETNN layers required to have at least one cell in $X_{G_1}$/$X_{G_2}$ whose receptive field is the whole set of nodes in $G_1$/$G_2$ is smaller than the number of EGNN layers required to have at least one node in $G_1$/$G_2$ whose receptive field is the whole set of nodes in $G_1$/$G_2$. This statement is exhaustively empirically confirmed in Table 3. Regarding the plot, including it in the main body of the paper is challenging due to space constraints. However, we will make an effort to incorporate it in the final camera-ready version of the paper.

Continuing response in the thread (References are included in the last part of the thread)

I appreciate the authors' detailed response, which I will now address:

[W1] I recognize that ETNNs can process combinatorial complexes, whereas EMPSNs are limited to simplicial complexes, making ETNNs more versatile. I also agree that the inclusion of two novel combinatorial complex benchmarks is a valuable contribution, highlighting the strengths of ETNNs. However, I believe the main advantage of ETNNs over EMPSNs is primarily technical, as both rely on scalarization combined with neighborhood-dependent updates. To my understanding, there is nothing inherently restrictive in EMPSNs update rule that prevents them from handling combinatorial complexes, even though they have been framed as a simplicial complex-focused architecture. Despite this, I agree that the ETNN update rule is more generic and flexible than EMPSNs.

Regarding the expressivity analysis, I agree that naturally extending the augmented geometric WL test is relevant and provides a non-domain-specific framework for analyzing geometric WL expressivity. However, the reduction of combinatorial complexes to Hasse graphs has been explored on multiple occasions and, in my view, applying a linear permutation-invariant map to the nodes of the cells appears to be a straightforward extension. Still the additional experiments testing ETNN on the counterexample structures from previous papers are a nice addition and the experimental section of the paper in general is strong.

[W2] I agree that the sum of pairwise distances is a natural and straightforward invariant to consider, and I appreciate the empirical demonstration that different aggregation methods do not impact expressiveness. However, from a theoretical perspective, I would have liked to see formal proofs for statements such as: “ETNN with permutation-invariant functions of pairwise distances can implement any function that ETNN with Hausdorff distance or convex hull volume can”. This would theoretically validate the empirical claim that different aggregation methods do not impact expressiveness.

Additionally, I believe that a graph could theoretically learn geometric node features for a node x, such as "the volume of the convex hull of the k-neighborhood of x" or "the distance of x from the barycenter of the graph." However, it is unclear whether either ETNN with permutation-invariant functions of pairwise distances or EGNN can effectively learn these types of node features.

[W3] I thank the authors for the clarification.

[W4] The authors note that “one can indeed lift a graph into a simplicial complex preserving the graph connectivity by taking triangles or cliques as higher-order simplices”. Given the paper's claim that using combinatorial complexes is more beneficial than simplicial complexes, a compelling theoretical result would be to demonstrate that ETNNs using lifts that result in non-simplicial complexes are more expressive than ETNNs using lifts that result in simplicial complexes. This might be a generalization of proposition 2, but I think it is an important one.

[W5] I thank the authors for their commitment to adding the requested experiments.

We thank the reviewer for their engagement, further feedback, and for stating that our paper deserves to be published. We sincerely and deeply appreciate the time the reviewer dedicated to our work. Here are some further comments.

[W1] It is undoubtedly true that ETNN is a simple equivariant generalization of CCMPNs, being a scalarization-based equivariant model, similar to EMPSN being a simple generalization of MPSNs, and EGNN being a simple generalization of MPNs. However, beyond the technical differences we already described, overall we showed that ETNNs are more general and flexible, more expressive, and similar or better performing while being hugely more scalable than SotA methods in the same class. These facts, together with the benchmark, software, and outreaching contribution of our work, should motivate its publication, as the reviewer agrees. This said, we already wrote in the future directions section that beyond scalarization-based architectures are of great interest. However, given that a gap in the literature was still present for scalarization-based models too, it made sense to us to first comprehensively fill it by working in the direction of ETNN.

[W2] We agree with the reviewer that open questions remain despite our extensive analysis and treatment. However, we believe this is almost always the case for any new architecture, especially in the case of a very young field such as Equivariant TDL (indeed, the same questions could be rightfully asked for the other architectures in the field). In this sense, ETNN, in our opinion, already represents the most comprehensive work in its related literature, and we believe it can spark interest to be further studied, by the community and by us as well.

Thanks again!

[W2] Although, as the reviewer agreed, it is unfeasible but interesting to add further theoretical results about this matter to this already extensive work, we, as we said, agree and believe that the theoretical and empirical impact of geometric invariants matters. For this reason, we conducted a small ablation study to analyze the impact of different geometric invariants on model performance. In this experiment, we used the property $\alpha$ of QM9 as the target variable. For the Combinatorial Complex configuration, we employed the first configuration outlined in Table 11 of Appendix J (which consists of atom, bond, and virtual cell with incidences and max adjacency). Regarding the choice of invariants, we utilized the following:

• Centroids Distance: This metric represents the Euclidean distance between the centroids of the two participating cells. It belongs to the second family of measures outlined in the manuscript, specifically the “Distances of permutation-invariant functions”.

• Hausdorff Distance: AS described in the paper. This measures the largest minimum distance between nodes in the two cells.

We indicate whether an invariant is active or inactive by setting it to 1 or 0, respectively.

Based on the results in the table, we make the following preliminary observations:

• Using both geometric invariants yields the best performance (Run 3).

• Excluding both Centroids Distance and Hausdorff Distance (no geometric information, CCMPN-like) significantly harms performance (Run 4).

• Hausdorff Distance appears to be a slightly better choice than Centroids Distance as a geometric invariant (Run 2 vs. Run 1).

These results show an interesting interplay among the invariants, giving hints on the impact of each one. As today is the last day to submit a revision of the paper, we plan and commit to extending this ablation study to include additional configurations and target variables for the camera-ready version of the paper.

As we mentioned in our previous message about the ablation study over the impact of the geometric invariant choice, we follow up with an extended study over multiple targets, and multiple experiment configurations (as these are reported in Appendix J. Below, we present the results of this study. The numbers of the experiment configurations correspond to the Combinatorial complex configuration, as presented in Table 11.

Remark: We note that due to time constraints we run the next configurations for a total of 350 epochs (instead of 1000 epochs of our original results in Table 11).

Experiment Configuration: 1

Experiment Configuration: 2

Experiment Configuration: 3

Experiment Configuration: 4

Following our preliminary results (see previous response), we validate again that:

1. Not using Centroids distance nor Hausdorff distance exhibits the worst performance.
2. In the majority of the runs, Hausdorff distance seems to perform slightly better than the Centroids distance.
3. Using both invariants usually yields the best performance.

We’d like to thank once again the reviewer for the constructive feedback, and we plan to include such analysis in the camera-ready version.

We sincerely thank the reviewer for their valuable feedback and suggestions. We would also like to thank the reviewer for pointing out as strengths the importance of this work for the TDL community, the text clarity, its clarity, completeness and self-containment. Next, we address each point raised as weakness and answer all of the reviewer’s questions.

Weaknesses.

[W1] Novelty

On the one hand, it is true that ETNNs are scalarization-based architectures, as EGNNs, on higher-order combinatorial domains, as EMPSNs. As such, they inevitably resound with each other.

However, we kindly disagree with the reviewer’s comment since ETNNs are a formal and more expressive generalization of both EMPSNs and EGNNs. For this reason, ETNNs unlock several features (e.g. arbitrary modeling of heterogeneous hierarchical higher-order interactions, tunable expressive power, and general learnable geometric invariants) that the constrained graph or simplicial structures of EGNNs and EMPSNs cannot accommodate (see Appendix C and F). As such, our framework can be used to design arbitrary E(n) equivariant TDL models. No other framework has the same power.

Moreover, although we believe that the generality of our framework is fundamental, we also recognized the necessity for applications whose exhaustive modeling (in a combinatorial/topological sense) is possible only via combinatorial complexes, and we introduced MolecularCCs and GeospatialCCs to tackle this problem (no other complex nor graph can model molecular and multi-resolution irregular geospatial data as straightforwardly as CCs). As a consequence, we achieved or matched SotA results among the Equivariant TDL models with a huge reduction in computational complexity.

As a byproduct, as Reviewer NQcw noticed too, our air pollution downscaling task represents a novel benchmark for the TDL community, addressing a need highlighted in the recent position paper [6]. Finally, we believe that the expressivity analysis is novel. In particular, our approach is, to the best of our knowledge, currently the most exhaustive one for scalarization-based equivariant TDL models, as it could be applied to analyze the (geometric WL) expressivity of any scalarization-based equivariant TDL model without the need for domain-specific coloring procedures.

[W2] Scalability

We kindly disagree with the reviewer since we believe that scalability is one of ETNN’s strengths (as pointed out by Reviewer NQcw too). In particular, as described in Section 5 and Appendix G, we showed that ETNN achieves or matches SotA results among E(n) Equivariant TDL models with considerable gains in memory, time, and, overall, scalability. These facts make ETNN amenable to be used on larger datasets in tailored, future works. The scope of this work is to introduce the framework, analyze it in detail, make it accessible, and show how it can be easily used for problems of vastly different natures. About the ablation, EGNN-graph-W has been introduced exactly to show how a graph model with the same parameter budget would perform, while the use or not of features has been extensively studied in Appendix J, Tables 10-11 (HO Features column). Finally, we do not consider an average improvement of more than 11% incremental.

Questions.

[Q1] Heterogeneous Interactions

The reviewer is right. Indeed, we state that CCMPNs can handle heterogeneity before we even introduce ETNN, and we give the right credit to [3]. However, it remains true that ETNN are set up for heterogeneity (in the geometric setting too). To improve the fairness of our presentation, in the revised manuscript we made clear that TDL models have this feature in the abstract.

[Q2] Path Complexes as CCs We believe that, as long as the considered paths are undirected (and this seems to be the case in [2], [3], [4] but not in [1]), a path complex can be easily cast as a CC in which the cells are the paths and the incidences and adjacencies are defined via the boundary relation of the complex (that, of course, takes into account the sequential nature of paths by definition) and not the set inclusion, as it happens in cell complexes (that indeed generalize path complexes). In the case of directed paths in the complex, then CCs cannot model it as the neighborhood structure would be possibly asymmetric. In this case, a different approach as the one in [5] should be employed. We also added [4] in our related works section.

Continuing response in the thread (References are included in the last part of the thread)

[Q4] Max Adjacency

We exhaustively study how the max adjacency (and the virtual cell) impacts the performance. In Tables 10-11, the “Adjacencies” column tells exactly what adjacencies are used. In general, the max adjacency and the virtual cell increase performance, and this is expected, as the virtual node [7] had a similar effect on graph-based models.

[Q5] Notation

The equations only require “containment” notation since $\mathcal{CN}$ contains neighborhood systems, $\mathcal{N}(x)$ contains neighboring cells, $x$ and $y$ are cells that contain atoms $z$.

Could the Reviewer please clarify which components of Equations 6 and 7 are they referring to?

[Q6] Higher-order geometric features

The setting that we tackle, which is the same as EGNN and EMPSN and is clearly stated at the beginning of Section 3, is the setting in which nodes (0-cells) are embedded in some Euclidean space, i.e., they come with both non-geometric and geometric features. This makes sense as higher-order cells are not necessarily physical entities. That said, in Remark 1 we also explain how to modify ETNN to integrate higher-order cells with geometric features, if available. Further, Appendix D discusses velocity-type inputs. However, in both cases, it is not clear to us what the reviewer means when they write about lifting (node) geometric features. The geometric features, as it is written in Equation 7, are updated only for the cells (say the nodes) that come with geometric features. On the other hand, the non-geometric features are updated for all the cells (Equation 6), and geometric invariants take into account the geometric features of all the nodes that are a part of a higher-order cell. Because of the message-passing operations between the different ranks, subsequent layers of message-passing at higher-order ranks will use the updated information from the geometric features at the node level.

References

[1] Grigor’yan, A.A., Lin, Y., Muranov, Y.V., et al. "Path Complexes and their Homologies." Journal of Mathematical Sciences, 248, 564–599 (2020).

[2] Truong, Q., & Chin, P. "Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes." Proceedings of the AAAI Conference on Artificial Intelligence, 38(14) (2024).

[3] Hajij, M., et al. "Topological Deep Learning: Going Beyond Graph Data." (2023).

[4] Li, L., et al. "Path Complex Neural Network for Molecular Property Prediction." ICML 2024 Workshop GRaM (2024).

[5] Lecha, Manuel, et al. "Higher-Order Topological Directionality and Directed Simplicial Neural Networks." arXiv preprint arXiv:2409.08389 (2024).

[6] Papamarkou, Theodore, et al. "Position: Topological Deep Learning is the New Frontier for Relational Learning." Forty-first International Conference on Machine Learning (2024).

[7] Sestak, Florian, et al. "VN-EGNN: E (3)-Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identification." arXiv preprint arXiv:2404.07194 (2024).

We thank the reviewer for the detailed response and for agreeing that our work is critical to the TDL field and starts filling an important gap in the literature. In the following, we address their follow-up comments. We hope that these further replies along with the overall positive feelings the reviewer showed for our work, can convince them to raise their score.

[W1, Q6] Novelty and Higher-order Geometric Features

Elaborating more about what we wrote in the previous reply, we would like to point out that:

(i) The architectural novelty is not representative of the novelty of our work as a whole. In this specific case, the novelty is given by:

• (a) Architectural novelty: ETNN improves on the two most important components of a scalarization-based equivariant architecture, i.e., feature updates and geometric invariants. Regarding features update, graphs and SCs induce specific neighborhood functions (node adjacency for the former, boundary, coboundary, up/down adjacencies for the latter), thus if a CC comes with a non-standard collection of neighborhoods, neither (ofc) EGNN nor EMPSN can handle it, both for non-geometric and geometric features updates. Regarding geometric invariants, EGNN/EMPSN uses tailored geometric invariants for graphs/SCs. Imagine applying EMPSN on a CC. Then, computing the volume of a non-simplex makes no sense. Similarly, pruning the pairwise distances based on the simplicial structure or using the angles of the planes induced by the dimension of the simplices makes no sense. On the other hand, ETNN can handle arbitrary neighborhoods, and its geometric invariants work on arbitrary CTSs (and formally generalize some of the invariants of EMPSN and EGNN).

• (b) Experimental novelty: the introduction of the new geospatial benchmark and our novel approach for molecular modeling represent a significant effort to tackle some of the important open problems of the field [3] and are a significant source of novelty (and an important resource for the community) as well.

• (c) Theoretical novelty: ETNN is the only available framework for designing arbitrary E(n) equivariant scalarization-based TDL models and the expressivity proof leveraging the novel notion of geometric augmented Hasse graph has not appeared before in the literature of equivariant TDL.

(ii) We believe that our architecture should be considered simple rather than trivial. If a model is designed to be general and flexible, and it is shown to be more expressive, scalable, and better performing than the SotA of its class, defining it trivial just because of the simplicity of its architectural definition sounds reductive (and slightly impolite) to us. This said, we explicitly wrote in the future directions section that beyond scalarization-based architectures are of great interest. However, given that the gap in the literature was present for scalarization-based models too, it made sense to us to comprehensively work in the direction of ETNN.

(iii) Higher-order relations are not just a generalization of adjacency matrices in graphs. In the context of ETNN, it is true that it is a widely general class of models on a broadly general space. However, E(n) Equivariant versions of architectures, e.g., [4], leveraging powerful concepts coming from algebraic topology, e.g., homology groups, stratifications, filtrations, etc., arise when ETNNs and CCs are particularized to specific combinatorial topological spaces, e.g., cell complexes. Similarly, E(n) Equivariant versions of architectures using specific insights on hypergraphs, e.g. [5], arise when ETNNs and CCs are particularized to hypergraphs. ETNN is the only framework having this joint feature, to the best of our knowledge.

Continuing response in the thread (references are at the end of the thread)

[W1, Q6] Cont. Regarding the geometric features for higher-order cells, we apologize if our previous reply was not clear enough. It is not true that “the geometric information is only inserted for cells that contain this information, and we rely on message passing to pass this geometric information to subsequent higher-order cells”. This is because geometric information can be leveraged either in the form of geometric features and/or in the form of geometric invariants. It is true that the geometric features are attacched only to cells that come with them, but the geometric invariants are used to update the subsequent (possibly) higher-order cells without any reliance on message passing, i.e. the geometric information contained in the geometric invariants is directly leveraged from the subsequent cells even if they do not come with geometric features. In this sense, what message-passing enables is the exchange of the (already attached) “geometric information” of all the cells, realized as geometric features and/or geometric invariants. This said, we totally agree that tailoring ETNN for applications in which higher-order cells have a (possibly) physical meaning with attached geometric features is intriguing, but it is an interesting future direction. Here, geometric invariants already represent a source of geometric higher-order information.

Regarding the geospatial task, we use the coordinates of the points (0-cells) as geometric features. We are really sorry we forgot to write it in Appendix I, it was not on purpose. We will add this specification in the next revision of the manuscript we are planning to submit in the next few days.

[Q2] Path complexes The reviewer is right about Lift 4b. It is not equivalent to a path complex, we will delete the sentence in the next revision to avoid adding further complexity. However, the reviewer's previous comment was about the possibility of rewriting a path complex as a CC. Regarding this, our answer remains valid: a CC where the cells (i.e., the elementary paths) are correctly specified and the boundary operation is used to define the neighborhood functions is equivalent to a path complex as in [2].

[Q5] Notation We believe they both correctly indicate containment. A cell $x$ is an element of the set of cells $\mathcal{X}$, a neighborhood function $\mathcal{N}$ is an element of the set of the neighborhood functions $\mathcal{CN}$ (i.e. the collection of neighborhoods). However, in the revision, we will make it clearer that $\mathcal{CN}$ is a set when we first introduce it before Equation 3.

References.

[1] Grigor’yan, A.A., Lin, Y., Muranov, Y.V., et al. "Path Complexes and their Homologies." Journal of Mathematical Sciences, 248, 564–599 (2020).

[2] Truong, Q., & Chin, P. "Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes." Proceedings of the AAAI Conference on Artificial Intelligence, 38(14) (2024).

[3] Papamarkou, Theodore, et al. "Position: Topological Deep Learning is the New Frontier for Relational Learning." Forty-first International Conference on Machine Learning. 2024.

[4] Battiloro, Claudio, et al. "Generalized simplicial attention neural networks." IEEE Transactions on Signal and Information Processing over Networks (2024).

[5] Bai, Song, Feihu Zhang, and Philip HS Torr. "Hypergraph convolution and hypergraph attention." Pattern Recognition 110 (2021): 107637.

I would like to thank the authors for the detailed response.

[W1. Q6] I apologize if my previous comment on simplicity of the model sounds impolite. I agree with the authors on the scope of the project, whose novelties focus on a flexible and a general pipeline for TDL. However, my main point still remains the same; it seems to me that your experiments cannot fully support the pipeline. However, I think it is worth a future research direction on defining geometric features for non-standard collection of neighborhoods. Given that your paper already brought many interesting insights and a novel benchmark, and my expectation may not be reasonable for an already informatively dense paper, I will consider to raise my score after the following concerns are resolved.

However, E(n) Equivariant versions of architectures, e.g., [1], leveraging powerful concepts coming from algebraic topology, e.g., homology groups, stratifications, filtrations, etc., arise when ETNNs and CCs are particularized to specific combinatorial topological spaces, e.g., cell complexes.

Could you reference relevant works on E(n) architectures that bridge the gap between Topological Deep Learning (TDL) and Traditional Topological Machine Learning methods that rely on homology groups, stratifications, filtrations, ... I checked [1], and it didn't mention anything about TDL and the traditional topological machine learning methods. Please kindly correct me if I am wrong. I think there is one concurrent work [2] trying to bridge the literature gap, but it seems the goal of your paper and this paper is very different. Because it is a concurrent work, so I just put it here as a reference and it doesn't affect the the paper score.

Regarding the geospatial task, we use the coordinates of the points (0-cells) as geometric features.

This is my main concern regarding your response. Even though coordinates play as an important geometric features for molecular graphs, I don't think it is the same for the novel benchmark. In many real world applications, if you compute geometric invariant features based on coordinates, it doesn't make a lot of sense. For example, housing price in big cities will be different from that in rural area; however, under the E(n) architecture, you are computing geometric invariant features and eliminate such a useful information. Can you elaborate more how coordinates of the points are appropriate geometric features for the benchmark?

[Q2] I completely agree with your argument; CC is the most general form of complex where one can define relations flexibly. I just meant that you should be more explicit on how you define Lift 4b to avoid confusion, as there are many complexes defined on paths.

[1] Battiloro, Claudio, et al. "Generalized simplicial attention neural networks." IEEE Transactions on Signal and Information Processing over Networks (2024).

[2] Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity. Under review at ICLR'25.

[W1. Q6] TDL and TML We thank the reviewer for bringing the reference to our attention. The goal of our previous reply was not to reference methods that bridge TDL with classical TML though, as reviewer’s comment was not about this. We wanted to stress that higher-order interactions are not just encoded as generalization of graph adjacencies, but can rather be leveraged using algebraic topology tools as well. In particular, the work in [1] we refer to as an example, (i) explicitly and formally uses (weighted) Hodge and Dirac theories, (ii) explicitly uses their corresponding spectral theories [2,3,4] (that can be leveraged in combination with filtrations [5]), and (iii) implicitly uses homology theory because the kernels of the Hodge Laplacian (and, as a consequence, the kernel of the Dirac operator) contain homology information as their dimensions are the Betti numbers of the complex. To link to our previous comment about this, E(n) Equivariant versions of architectures like [1] can be readily obtained from our ETNN framework.

In terms of macro-areas, works like [1] help to bridge TDL with algebraic topology using more of a Topological Signal Processing (TSP) [6] perspective, while we agree that works like [7] help to bridge TDL with algebraic topology using more of a TML perspective. Furthermore, works like [8] help to bridge TDL with equivariant DL using more of a TML perspective, while a general work like ETNN helps to bridge TDL with equivariant DL implicitly using more of a TSP (and traditional relational DL) perspective. Overall, the three mentioned disciplines (TSP, TDL, TML) are surely overlapping, fall under the same umbrella, and many links among them are yet to be studied (e.g., further insight on if and how results from [7] applies to proper convolutional models like [1] that explicitly take some topological invariants into account). However, this surely is an exciting motivation for being in this field at this time. Finally, we will also cite [7] in our paper as it is surely relevant to our work.

[W1. Q6] Geospatial task coordinates as geometric features We would like to clarify some potential misunderstandings. Most coordinate systems are, by construction, artifacts built with notions of symmetry and equivariance, as they are often derived from projections. Specifically, we use the Mercator projection, which uses meters as the unit scale. This projection has the key property that Euclidean distance in coordinate space corresponds to geodesic distance. Thus, E(n)-equivariance is implicitly essential to the purpose of the Mercator projection. The exact coordinate values are irrelevant; what is important for our task is the projection's usefulness in expressing geodesic distances. Consequently, it is never desirable for the model to behave erratically with respect to a Euclidean transformation of the Mercator coordinates.

In light of the above remark, it is not true that geometric invariant features eliminate useful information. It is known that even just pairwise distances are sufficient to recover angles in Euclidean space (cf., [9; Appendix E]), and therefore a coordinate system modulo the Euclidean group. The same logic applies to our study: by using geometric invariants in geospatial coordinate systems, we allow the model to use the coordinates modulo Euclidean transformations, which is the original intention of the Mercator projection. This implies that the model can still capture location-specific random effects in node-level tasks using only invariants, which is the reviewer's main concern.

We recognize that this is a confusing point and central to the paper. Therefore, we have added a summarized version of our response to the revised text.

[1] Battiloro, et al., 2024. "Generalized simplicial attention neural networks." IEEE Transactions on Signal and Information Processing over Networks.

[2] Calmon et al. 2023. "Dirac signal processing of higher-order topological signals." New Journal of Physics 25.9.

[3] Hansen & Ghrist, 2019. Toward a spectral theory of cellular sheaves. Journal of Applied and Computational Topology.

[4] Yang, et al. 2022. "Simplicial convolutional filters." IEEE Transactions on Signal Processing.

[5] Grande & Schaub, 2024. "Disentangling the Spectral Properties of the Hodge Laplacian: not all small Eigenvalues are Equal." ICASSP.

[6] Barbarossa & Sardellitti 2020. "Topological signal processing over simplicial complexes." IEEE Transactions on Signal Processing.

[7] Eitan et al., 2025. "Topological blind spots: Understanding and extending topological deep learning through the lens of expressivity.", under review at ICLR'25.

[8] Verma et al., 2024. "Topological Neural Networks go Persistent, Equivariant, and Continuous." ICML.

[9] Satorras et al., 2021. "E(n) Equivariant Graph Neural Networks." ICML.

We appreciate the thorough reviewer’s constructive comments and the valuable questions. We would also like to thank the reviewer for pointing out as strengths the transparency of our experiments, the text clarity, the novelty of the geospatial task, and the computational efficiency of ETNN. Next, we address each point raised as weakness and answer all of the reviewer’s questions.

Weaknesses.

[W1] TDL vs GDL: novelty as a conceptual framework

As long-time practitioners of TDL (and GDL) we the authors completely understand the point raised by the reviewer. However, there are multiple answers to why TDL is a motivated framework, spanning from (i) more conceptual (about the framing of TDL and GDL as fields in the ML landscape) to (ii) more pragmatic (about the usage of TDL models in real-world applications).

About (i), the quick (but not exhaustive) answer is that GDL and TDL stem from two formally and inherently different perspectives on the same objects, “non-Euclidean spaces”. GDL (in the sense of [1]) is built on group-theoretic arguments along with the frequent usage of Hilbert Spaces (strictly related to manifold learning and, in general, to metric spaces), while TDL is solely built on the modeling assumption of data living on the neighborhoods of a combinatorial and/or topological space and having a relational structure induced by the neighborhoods’ overlap. As such, TDL works tend to put more emphasis on what graph (or manifold) based models struggle to capture, e.g., higher-order combinatorial interactions. Overall, the intersection of GDL and TDL is clearly not empty, but we believe that both fields have a well-framed conceptual motivation. Further insights can be gained from the thesis in [2]-[3].

About (ii), we firmly believe that TDL as a field needs stronger empirical evidence of its effectiveness. For this reason, in this work, we focused on proposing applications in which higher-order interactions (a) matter and (b) can be captured via ETNN (and some other TDL models in general). Indeed, and this is also a partial answer to [Q2] below, the aim of the proposed molecular CC and the novel geospatial benchmark is exactly to prove (a)-(b) true and, more, to also show that there are situations in which neither graphs nor higher-order combinatorial topological spaces (SC or CW complexes) can leverage the available information in a jointly exhaustive and computationally efficient way as CCs do. We thus prove that the modeling of higher-order interactions through CCs indeed offers benefits, especially for structured hierarchical data (e.g., molecular and geospatial data).

Finally and most importantly, to the best of our knowledge, ETNN is currently the most exhaustive framework for merging scalarization-based equivariance arguments from GDL with pure TDL arguments.

[W2] ETNN vs EGNN

We kindly disagree with this observation since the aim of Proposition 1 is not to show that ETNN is equivalent to some specific EGNN, but rather the opposite: to show that some specific ETNN is computationally equivalent to a (non-standard) EGNN over a Geometric Augmented Hasse Graph. As such, the considered subclass (that is the easiest possible) is not representative at all of the whole class of architectures that can be derived from the ETNN framework. EGNN is formally a particular case of ETNN. The “graph-based” method we introduce is only useful to easily and elegantly prove expressiveness in the geometric WL sense, which is dependent only on the relational structure and the geometric features induced by the CC, not on the specific ETNN architectural choice. This is an intuition already proved successful in works like [4] for non-geometric settings.

[W3] Classical Topology in TDL

This is a key comment (for this work, its novelty, and TDL in general). We agree that the inclusion of domain-specific data along with the design of a CC (not the graph) that can directly exploit them in a principled way, is doing most of the work. However, given our reply to [W1], we do not believe that this should not be considered topological (in a non-classical sense), as it is the higher-order relational structure induced by the underlying space that contributes to the effectiveness of ETNN. That said, the formal concepts coming from classic algebraic topology, e.g., homology groups, stratifications, filtrations, etc., arise when CCs are particularized to specific combinatorial topological spaces. Several works, e.g., [5]-[6], have shown improvements related to classical topological arguments. E(n) Equivariant versions of these architectures can be directly derived from the ETNN framework.

Continuing response in the thread (References are included in the last part of the thread)

[Q4] Definition of Geospatial CC

Thank you for your careful reading and for bringing this omission to our attention. The corrected notation should indeed be $s: X \to \mathcal{P}(T)$.

Regarding your second comment, we have revised the manuscript to include a more concrete definition of a geospatial CC (GCC) at the beginning of the relevant section. GCCs are now explicitly defined as CCs in which all cells are subsets of a geospatial domain. However, we believe that the mapping $s$ remains necessary for a more formal definition and to explain the construction of neighborhood systems.

We have also clarified the definition of a polyline in the revised manuscript. It is now stated that a polyline is simply a sequence of connected lines, and we have included a citation to an introduction to common data formats in geographic information systems. We hope that these improvements to the wording highlighted in the revised manuscript provide greater clarity.

[Q5] Combinatorial topological modeling of multi-resolution irregular geospatial data

As we explain in Appendix H.2, TDA methods have been applied to geospatial data (we added the missing references among the ones indicated by the reviewer). However, TDA is different from TDL (see again Appendix H.2), and in all the referenced works there is no combinatorial characterization induced by, e.g., political partition and no multi-resolution. They are mostly all variants of PH-related arguments applied to geospatial data.

[Q6] Variances in Table 2.

We originally did not report variances since our experiments are averaged over 3 seeds, which makes variance calculations unreliable. We are now working on running with additional seeds and will report the results in the next few days when the experiments are complete.

[Q7] Line 242

All the geometric invariants are geometric objects. However, their definition is totally dependent on the underlying topological combinatorial space. E.g., which are the pairwise distances to be summed are given by the relational structure of the space and its neighborhoods.

References

[1] Bronstein, Michael M., et al. "Geometric deep learning: Grids, groups, graphs, geodesics, and gauges." arXiv preprint arXiv:2104.13478 (2021).

[2] Bodnar, Christian, “Topological Deep Learning: Graphs, Complexes, Sheaves”, PhD Thesis, https://www.repository.cam.ac.uk/items/06b0b8e5-57d1-4120-8fad-643ce4d40eda (2022).

[3] Battiloro, Claudio, “Signal Processing and Learning over Topological Spaces”, PhD Thesis, https://theses.eurasip.org/theses/974/signal-processing-and-learning-over-topological/ (2024).

[4] Jogl, Fabian, Maximilian Thiessen, and Thomas Gärtner. "Expressivity-preserving GNN simulation." Advances in Neural Information Processing Systems 36 (2024).

[5] Yang, Maosheng, Elvin Isufi, and Geert Leus. "Simplicial convolutional neural networks." ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022.

[6] Battiloro, Claudio, et al. "Generalized simplicial attention neural networks." IEEE Transactions on Signal and Information Processing over Networks (2024).

[7] Sestak, Florian, et al. "VN-EGNN: E (3)-Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identification." arXiv preprint arXiv:2404.07194 (2024).

​​Standard errors for the geospatial task and additional seeds

Dear reviewer, we have conducted multiple additional runs for the geospatial task and computed standard errors based on your suggestion. We ran 10x more seeds (30 in total). Despite the high variance, the results are very similar to those reported in the paper. Standard errors are computed with the formula $\sigma/\sqrt{n}$, which is recommended when having at least sample size 30.

In light of the extension of the discussion deadline, we would like to thank the reviewer again for the already positive assessment of our work and for their engagement, and ask if there is anything else we can do to further improve the manuscript and its score.

We also kindly invite the reviewer to parse the discussions we had with the other reviewers, as we believe that the overall engagement and the individual feedback have clarified or improved several aspects of the paper.

Thanks a lot!

References.

[1] Bronstein, Michael M., et al. "Geometric deep learning: Grids, groups, graphs, geodesics, and gauges." arXiv preprint arXiv:2104.13478 (2021).

[2] Bodnar, Christian, “Topological Deep Learning: Graphs, Complexes, Sheaves”, PhD Thesis, https://www.repository.cam.ac.uk/items/06b0b8e5-57d1-4120-8fad-643ce4d40eda (2022).

[3] Battiloro, Claudio, “Signal Processing and Learning over Topological Spaces”, PhD Thesis, https://theses.eurasip.org/theses/974/signal-processing-and-learning-over-topological/ (2024).

[4] Papamarkou, Theodore, et al. "Position: Topological Deep Learning is the New Frontier for Relational Learning." Forty-first International Conference on Machine Learning. 2024.