We sincerely thank the review for their thorough review and feedback!

We will now address the points raised in the review:

I do not fully understand the “learning” the representation here, because the representation is not particularly being learnt. It is being computed by using the persistent homology of the point cloud

There seem to be different views of what should be considered as “learning” in the community. The English wikipedia article on feature & representation learning explicitly lists unsupervised algorithms like PCA, k-means, and LLE, which “learn” the representation by performing specific computations. Topological Point Features fit in well with this view on representation learning. However, we want to be very clear in what TOPF and what it does not do. Thus we have added a clarification on this to the introduction and thank the reviewer for their valuable remark!

Experimental evaluation is limited to clustering. And even in clustering, it is primarily limited to shapes which are partially/fully topologically spherical.

In the qualitative experiments, we reported the feature-level vectors instead of the received clusterings. (See Figure 4 & 8)

[Same as author rebuttal:] We agree with the reviewer that having more experiments is always a great thing! We have now added an experiment were we use TOPF to extract topological features on the embedding space of a variational autoencoder trained on real-world image data and show that these point-level features correspond to the underlying topology of the data acquisition process. (See Figure 2 of the attached pdf.) Already in the pre-Rebuttal-phase, the non-main-text (exluding paper checklist) was longer than the main text of our paper. Downstream-Applications in TDA usually require a lot of additional explanation and description of the experimental setup and are significantly more complex than running simple off-the-shelves benchmarks, as is the case in many other areas of Machine Learning. This is why in many cases, the introduction of new methods and the application on real-world data are even divided into two different papers. Because of these reasons, we believe that adding experiments on top of our introduction of a new Clustering Benchmark Suite (TCBS), comparison with existing methods on TCBS, experiments on real-world protein data and on state spaces of dynamical systems, on the latent space of a Variational Auto Encoder, and on robustness with respect to noisy data, robustness wrt. to hyperparameter choices, and on run-times on growing point clouds would be out of scope for a single NeurIPS paper that introduces a theoretically novel method for extracting topological point features.

The robustness of the approach is due to the robustness of harmonic persistent homology known in the literature

Generally, we consider this to be a strength. However, we want to highlight that, to the best of our knowledge, this is the first work to utilise this robustness in a novel method. To the best of our knowledge, the only other prior experimental work using harmonic persistent homology [1] always construct simplicial complexes at the birth of the feature and extract different features from this than we do. The SCs at the birth of the features are unstable.

The paper uses well-known notions in the TDA literature in the context of point-clouds, which amounts to an incremental progress in this direction.

We want to emphasise that doing TDA and relating the global topology back to the local individual points is a very novel idea in TDA with very limited prior work. Although our work builds on already established concepts and theory, it combines these pieces in a novel way that has not been done before.

It would strengthen the paper if the authors include a small paragraph explaining why projecting onto the harmonic subspace solves the problems that exist in using the homology representatives directly.

Thank you very much for this suggestion! We will happily add a paragraph on that! Please note that we already provide some explanation on this in lines 164--175. There are two main reasons that projecting onto the harmonic space is a good idea:

1. All of the countless [2] representatives representing the same homology class get projected onto the same harmonic representative. Thus, harmonic representatives form a canonical and unique homology representation. (This can be explained by the Hodge Laplacian and the fact that $\ker L_k(X) \cong H_k(X)$.
2. The harmonic representatives minimise the energy among all possible representatives. In other words, this ensures smoothness of the representatives and assigns every simplex a value that corresponds to how much it contributes to the homology class. (The precise mathematical formulation is simply the minimisation of energy.)

We will explain this in more detail in the appendix! Thank you again for the suggestion.

‘Spaces in topology are “continuous”’. Continuity is a notion defined for functions on topological spaces and not for topological spaces themselves. Spaces are connected.

Because "continuous" has no mathematical definition for spaces, we intended to use it to convey an intuition. "Connected" is suboptimal, as even a two-point set can be connected if equipped with the trivial topology. However, to avoid confusion we will stick with "connected". Thank you again for noticing!

Can this approach be used for other point-cloud related machine learning tasks?

We believe that topological features can be used in many applications with data with topological structure. We have added an experiment on the interpretability of latent spaces of VAEs. (Figure 2 of the additional pdf)

[1] Davide Gurnari, Aldo Guzmán-Sáenz, Filippo Utro, Aritra Bose, Saugata Basu, and Laxmi Parida. Probing omics data via harmonic persistent homology.

[2] Actually, this is still a finite number as we are working with finite simplicial complexes.

I would like to thank the authors for their efforts and response. I have read the responses and have adjusted my score.

We sincerely thank the reviewer for their thorough review and feedback! We believe we have significantly improved the paper based upon your comments! We will address the individual points:

"Regarding unjustified claims":

1. By higher-order we mean of order 1 and above in the sense of homology. In this interpretation, our claims are true and we are not underselling existing approaches, as cluster structure is only 0-th order homology structure. However, we want to be as clear and fair as possible. We have revised the sentences for clarity and state the differences more clearly.

2. This is not quite a strong claim as it might seem. Ordinary cluster structures are basically captured by noisy 0-spheres, 1-dimensional homology appears almost always in the form of some noisy deformed cycle, and so on. Even very basic non-sphere structures like tori appear only in very rare circumstances in real-world data, as is evidenced by the existing applied TDA literature. Finally, we explicitly did not claim this for all datasets and thus feel justified.

Regarding Theory

1. It is true that Theorem 4.1 applies in a restricted scenario. The purpose of Thm 4.1 is to verify that TOPF does provably correct things on ideal data sets recovering the implanted topological structure.
2. Thank you very much for reading the proof! :) Harmonic cycles (as those considered by topf) indeed take absolute values between 0 and 1 almost always on the majority of the simplices. This is, unless they don't have parallel simplices around the same generalised hole: In this case, it is not possible to minimise the energy by distributing the "flow" (in the 1-dimensional case) among the parallel simplices and a single simplex needs to account for all the contribution to homology. This is exactly the case in the theorem: Because the points lie on a perfect n-sphere with radius 1, the first n-simplex appears only at point 1 in the alpha-filtration. (In a VR filtration, this would not hold.) Without any n-simplices, there are no "parallel" (n-1)-simplices, and thus every simplex has either value -1 or 1 (depending on orientation) or 0 (Or a normalised version of this). We will carefully describe this argument in the proof.

Regarding the methodology

• The example given by the reviewer does not underline their claim: In Z/3Z coefficients, a homology representative for the triangle needs to assign every edge a value such that the oriented sum (*+1 at the tip of the edge and *-1 at the tail) at each of the nodes is equal to 0. If we fix the orientations of the edges to all be clockwise, this is only possible with all edges having the same value. All edges having a 0 obviously does not work, so (1,1,1) and (-1,-1,-1) remain as options. In the same setting, these are generators for real-valued homology as well.

In general, it is true that being a generator of Z/p-homology does not guarantee being a generator of homology with real coefficients. However, there are multiple reasons why this is not a problem in practice.

1. The homology generators computed by persistent homology have a very special structure. In dimension 1, instead of being some random formal sum of edges, they form a sum of closed cycles. In higher dimensions, similar statements hold. This however then means that the PH generators in Z/3Z coefficients will always be in the kernel of the boundary operator in R coefficients.
2. Now the only thing that can happen is that the the generator is in the image of the boundary in R coefficients. This is precisely the case when the represented homology class has p-torsion. On real-world data, this very rare. (This is mainly due to the fact that manifold needs to be "very complicated" to have torsion, whereas torsion homology groups are easy to achieve. E.g., RP^2 (the most simple space with torsion in the homology group) or can only be embedded in R^4 or above).
3. Furthermore, it does not suffice for the space to have a homology class with some torsion, only homology classes with 3-torsion will be relevant.
4. We can very easily check whether this is the case: If the projection of the generator to the harmonic space, which we compute anyway, vanishes, the considered homology class has torsion, and thus we can safely exclude it from our analyses. We will add a check for this in our implementation.
5. We are not throwing any harmonic information away this way: all homology classes representable by harmonic forms don't have torsion (in Z-coefficients), and all homology classes without torsion appear in Z/3Z homology.

We will add a version of this explanation to the appendix, thank you very much for pointing this out!

• We run spectral clustering on the feature vectors produced by TOPF to obtain a clustering of the points. (k-means or similar clustering algorithms would work as well.) We have added an explanation to the experiments section and the caption of Table 1. Thank you for catching this!

• The purpose of Fig. 4 is to show that TOPF corresponds roughly to our intuition of relevant topology: In a), the features detect some of the pockets (green, black, purple) of the protein and the hole in the middle (red), in b) the loops in the protein structure are recovered, and in c) the two regimes of the Lorentz attractor are recovered. Because there is no ground truth for this, we can't report an ARI, which we did in the quantitative experiments.

• We have added experiments on the performance of TOPF on TCBS for a wide range of hyperparameter settings. (Fig. 1 of the pdf). In summary, TOPF is not very sensitive to hyperparameter changes around the default values, making hyperparameter choices robust. We have picked the hyperparameters for TOPF to perform well across the large range of applications in our paper. We will add the experiments to the appendix section discussing the hyperparameters.

We apologise, but we will answer the remaining questions in a comment.

Thank you for your responses.

• "Harmonic cycles (as those considered by topf) indeed take absolute values between 0 and 1 almost always on the majority of the simplices." This is not what your theorem says. It says that it takes values in {$0,\pm 1$}, which still seems wrong to me.
• "parallel simplices around the same generalised hole". What is a "generalized whole"? I did not understand this part of the discussion.
• "We run spectral clustering on the feature vectors produced by TOPF". How were the parameters selected?
• I am not convinced by your argument for lifting Z/3Z cocycles to Z-cocycles. Indeed, the original paper that proposed to get circular coordinates out of persistent homology computations [1] has a whole section on this, "2.4 Lifting to Integer Coefficients". In particular, they say that when the "easy" lift that you use fails, they don't have particular guidance as to how to proceed: "This is all very well. Unfortunately, the equation η = d1ζ is a Diophantine linear system. At present, we can provide no particular guidance as to how to solve the system (other than by vague appeal to off-the-shelf Diophantine or integer linear programming solvers), even if we know that a solution exists." Since the publication of that paper, the software DREiMac has been published. They do address this issue using a linear programming solver. See the documentation at [2], where they also have an example showing how things can fail. In particular, that example shows that torsion is not the only issue that can occur, as that data has no torsion.

[1] Persistent Cohomology and Circular Coordinates. Vin de Silva, Dmitriy Morozov & Mikael Vejdemo-Johansson. https://link.springer.com/article/10.1007/s00454-011-9344-x

[2] https://dreimac.scikit-tda.org/en/latest/notebooks/parameters_prime_and_check_cocycle_condition.html

We sincerely thank the reviewer for their feedback! We are happy to read that they find the paper sound and well-presented.

Topological features are usually not localized, the idea of being able to bring back the topological descriptor to the relevant points is quite novel and impactful.

We are very excited about this as well! :)

the feature selection is very heuristic.

Topological features can have very different meaning and significance based on their context. Thus it does not seem plausible that there exists one and only one provably best and correct way to select features. The approach by TOPF takes a view based on concepts of Algebraic Topology, Differential Geometry, and TDA to select the most significant features, which seems to work well in a variety of applications. Given a lot of training data for a particular case, we could of course train a neural network to select the most relevant features in this applications. This is a promising future work, but because this would require an entirely new architecture on top of TOPF we believe this would not fit well into the current paper. Furthermore, we have conducted additional experiments on the robustness of TOPF wrt. hyperparameter choices, see Figure 1 of the attached pdf.

The evaluation is only on point cloud clustering. Since we are evaluating effectiveness and robustness of localized features, feature/point correspondence problems would have been interesting.

This sounds like a very interesting problem and exciting application, thank you very much! We understand that in a nutshell, the current SOTA methods use neural networks trained on certain sets of features. As this probably requires coming up with a (at least in some part) new architecture, training the model, etc., this sounds like an entirely new additional idea, which would suggest featuring it in a follow-up paper.

[Same as in author rebuttal:] We agree with the reviewer that having more experiments is always a great thing! We have now added an experiment where we use TOPF to extract topological features on the embedding space of a variational autoencoder trained on real-world image data and show that these point-level features correspond to the underlying topology of the data acquisition process. (See Figure 2 of the attached pdf.) Already in the pre-Rebuttal-phase, the non-main-text (exluding paper checklist) was longer than the main text of our paper. Downstream-Applications in TDA usually require a lot of additional explanation and description of the experimental setup and are significantly more complex than running simple off-the-shelves benchmarks, as is the case in many other areas of Machine Learning. This is why in many cases, the introduction of new methods and the application on real-world data are even divided into two different papers. Because of these reasons, we believe that adding experiments on top of our introduction of a new Clustering Benchmark Suite (TCBS), comparison with existing methods on TCBS, experiments on real-world protein data and on state spaces of dynamical systems, on the latent space of a Variational Auto Encoder, and on robustness with respect to noisy data, robustness wrt. to hyperparameter choices, and on run-times on growing point clouds would be out of scope for a single NeurIPS paper that introduces a theoretically novel method for extracting topological point features. However, we are excited to use and see others use TOPF for new applications.

We thank you for your time to read our rebuttal! We are excited to hear back from in the discussion phase!

Dear reviewer,

Thank you again for your review and suggestions! In the rebuttal, we have added a new experiment on the embedding space of variational autoencoders, validated the robustness of TOPF with respect to various hyperparameter changes experimentally, studied the runtime on point clouds with growing $n$, and added a section on the relationship between Hodge theory, differential forms, and topf and the theoretical intuition behind our algorithm (See the rebuttal to reviewer Zmis). In particular, we hope to have addressed the points raised by the reviewer in our rebuttal above.

As the author-reviewer discussion period closes in ~ 24h, we would be happy to answer any further questions or receive any more comments on our work and our rebuttal. In particular, we would be grateful to hear whether the reviewer feels that we have addressed the points raised by them.

Thank you in advance for your reply!

We sincerely thank the reviewer for their review! We are happy to hear that they find our work well-written and novel.

it would be beneficial to include additional evaluation, qualitative and quantitative, on real-world data and additional applications

[Cf. Authore rebuttal:] We agree with the reviewer that having more experiments is always a great thing! We have now added an experiment were we use TOPF to extract topological features on the embedding space of a variational autoencoder trained on real-world image data and show that these point-level features correspond to the underlying topology of the data acquisition process. (See Figure 2 of the attached pdf.) We believe this to be an interesting contribution to the field of interpretable AI. Already in the pre-Rebuttal-phase, the non-main-text (exluding paper checklist) was longer than the main text of our paper. Downstream-Applications in TDA usually require a lot of additional explanation and description of the experimental setup and are significantly more complex than running simple off-the-shelves benchmarks, as is the case in many other areas of Machine Learning. This is why in many cases, the introduction of new methods and the application on real-world data are even divided into two different papers. Because of these reasons, we believe that adding experiments on top of our introduction of a new Clustering Benchmark Suite (TCBS), comparison with existing methods on TCBS, experiments on real-world protein data and on state spaces of dynamical systems, on the latent space of a Variational Auto Encoder, and on robustness with respect to noisy data, robustness wrt. to hyperparameter choices, and on run-times on growing point clouds would be out of scope for a single NeurIPS paper that introduces a theoretically novel method for extracting topological point features. However, we are excited to use and see others use TOPF for new applications.

Additionally, comparison with other well performing modern machine learning methods, such as graph neural networks for point cloud clustering, needs to be discussed, for completeness.

TOPF is an unsupervised feature extraction algorithm, whereas GNNs require training data and labels. Thus, using GNN on the TCBS is not possible. We have compared against the to us most-relevant non-supervised algorithms. If you believe an important non-supervised algorithm is missing, we would be happy including it in the final paper.

What is the runtime of the proposed method? How does it change with the point cloud size and does it have limitations on the size of point cloud that can be processed with it?

We report the runtime of the proposed method in Table 1. All the examples of the TCBS run in under 40s on a modern laptop. Furthermore, we have added experiments on how the runtime changes when increasing the number of points while preserving topology, (Figure 3 of the additional pdf).

We hope to have convinced the reviewer of the soundness of our methods. We want to emphasise that doing TDA and relating the global topology back to the individual points is a very novel idea with very limited prior work. TOPF is not just simply a slightly different algorithm among countless similar ML approaches.

Thank you for reading our rebuttal, we would be very happy to answer any questions or hear back from you!

I would like to thank the authors for their efforts and response.

Already in the pre-Rebuttal-phase, the non-main-text (exluding paper checklist) was longer than the main text of our paper.

I don't believe the length of the paper can or should be used as a justification for missing experiments if these are necessary to fully appreciate the contribution of the paper. Specifically for this submission, experiments with data beyond specifically designed synthetic dataset, are necessary to validate that the proposed approach can be applicable for data beyond such a dataset - if it is not the case, this would greatly limit the potential of the proposed approach. I would like to thank the authors for adding the new experiments with the embedding space of a variational autoencoder.

If you believe an important non-supervised algorithm is missing, we would be happy including it in the final paper.

Some examples of possibly relevant unsupervised methods in image segmentation area are "DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering" by Aflalo et al. (specifically for GNNs) or "Unsupervised semantic segmentation by distilling feature correspondences" by Hamilton et al. It would be interesting to understand whether the above approaches designed for images, or any similar unsupervised approaches, can be applied to the proposed benchmark and how they would compare with the proposed approach.

Thank you very much for your response!

Specifically for this submission, experiments with data beyond specifically designed synthetic dataset, are necessary to validate that the proposed approach can be applicable for data beyond such a dataset - if it is not the case, this would greatly limit the potential of the proposed approach.

We agree with the well-written response of the reviewer in this regard! We just want to highlight that we already include experiments on data from synthetic datasets, on protein atom coordinates, on state spaces of dynamical systems, and now on embedding spaces of variational autoencoders.

Some examples of possibly relevant unsupervised methods in image segmentation area are [...]

Thank you for your suggestion! As we understand it, the two presented image segmentation benchmarks work on image data, where every pixel in a grid has an (R,G,B) value. The datasets of our benchmark suite are point clouds in $n$-dimensional space without any specified or grid-like structure. While it might be possible to turn an image into a point cloud in 5D, the other way does not work. Thus it is not possible to use image segmentation algorithms as a baseline for the Topological Clustering Benchmark.

Thank you again for this discussion, we will be happy to answer any further questions!

We once again want to thank the reviewer for encouraging us to explore additional baselines for the Topological Clustering Benchmark Suite!

If we understood the reviewer correctly, they were interested in how pretrained neural network architectures would perform on the introduced Topological Clustering Benchmark Suite. A good baseline for 3d point cloud segmentation tasks is Pointnet [1], which is a neural architecture that directly deals with unstructured point cloud data.

We have pretrained Pointnet on the ShapeNet-Part Segmentation dataset [2]. We trained for 200 epochs taking ~4 hrs on 2 nVidia L40 GPUs achieving an accuracy of 0.93 and an IoU of 0.83 corresponding to the values expected from the literature. ShapeNet-Part is a dataset of 3d objects corresponding to 16 diverse categories. In the segmentation task, the neural network learns to divide the point cloud into different segments corresponding to different semantic parts. We have then evaluated the performance of the pretrained pointnet on the datasets of the TCBS across all categories. We report the highest ARI (Adjusted Rand Index) for every dataset:

This shows that TOPF outperforms the above-introduced Pointnet version on the TCBS. This is as expected, as the neural network was not trained specifically on the datasets of the test set. However, as there is very scarce data and thus no training for the extraction of topological features, it is to be expected that all pre-trained neural network architecture will suffer from similar performance on TCBS. We note that pointnet is not the current SOTA on Shapenet-part-Segmentation. However, pointnet is still in the range of 6% IoU of GeomGCNN and thus representative of the general capabilities of deep learning models, whereas the mean performance difference to TOPF on TCBS was 42%.

We thus believe to have addressed the reviewer's request for a pre-trained neural network baseline, comparing a pretrained version of Pointnet with the performance of TOPF on our dataset. TOPF outperforms PointNet by a wide margin of 42% ARI.

[1] Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652-660).

[2] https://shapenet.org

We thank the area chair for engaging in the discussion! We will provide replies to the comments below:

Questions:

In Theorem 4.1, does the condition "Let X consist of at least (n + 2) points (denoted by S) sampled uniformly at random from a unit n-sphere in R^{n+1} and an arbitrary number of points with distance of at least 2 to S" allow us, for n=1, to consider 3 random points in the unit circle in the plane?

Yes, this is a possible setting of the theorem.

If yes, and these three points form an obtuse triangle, how can we get a non-trivial cycle in 1D homology of the alpha-filtration?

The scenario mentioned above results in a non-trivial cycle in 1d homology, just as our theorem predicts. We will briefly explain why: In an alpha-filtration, the birth of a simplex is determined by the radius of a circumscribed sphere. For 1-simplices (edges) with length l, a "circumscribed" 0-sphere (consisting of just the 2 vertices of the edge) has radius$l/2$. With probability 1, the random points on the circle will not lie exactly opposite each other, and thus have a distance strictly smaller than 2. Thus, the respective three 1-simplices (the faces of the triangle) have a filtration value strictly smaller than $2/2=1$ with probability $1$. This is not changed by the triangle being obtuse. All of the vertices of the 2-simplex lie on the unit sphere with radius $1$. Hence the radius of its circumcircle and its filtration value is $1$. Let L be the maximum distance between two of the points. Then, there is a non-trivial cycle in 1d-homology in the non-empty interval $[L/2,1)$.

Limitations

Line 240 says that "We found empirically that a thresholding parameter of delta = 0.07 works best across at the range of applications considered below". Such a manually chosen threshold is unlikely to be universal for applications in the same sense as (for example) the gravitational constant, which is actually expressed via Earth's mass, in Newton's law of gravity.

We want to emphasise that virtually all ML methods have hyperparameters and that we have shown that our hyperparameter choice offers good performance across a range of application. Even more importantly, we have shown in experiments in the additional pdf that the performance of TOPF is robust against medium changes of delta. As explained in line 239, delta is not the final thresholding value, but rather a constant to adapt the thresholding parameter to the simplicial complex and the current harmonic representative. We believe this to be well in line with what almost all modern ML algorithms do.

Reinterpretation of the gravity metaphor

Furthermore, we would like to offer our own interpretation of the AC's metaphor. First, we need to fix some notation: We assume that the "gravitational constant" the AC is talking about actually refers to small g, the local gravitational field (of the earth). Then, g is related to the mass of the earth and the radius via the gravitational constant $G = 6.6743... \cdot 10^{-11} N⋅m^2/kg^2$. In our paper, we threshold values above $T=\delta*X(e,S)$ where $X$ is some expression depending on the simplicial complex S and the select harmonic representative S (See line 239). In the above metaphor, T plays the role of the local gravitational field, as it depends on other variables. Our thresholding parameter delta however is characterising this dependency of $T$ on $X(e,S)$. In the analogy, this represents the gravitational constant $G$, which is really just a random constant [1] of the universe. We have already taken care to adapt the thresholding procedure to the relevant situation, and thus we believe that we have, given the experimental evidence, reasonable hope that this set-up generalises well. We believe this to be well in line with what almost all modern ML algorithms do.

Thank you again for your questions and comments! We will be happy to answer or discuss any follow-up questions in the discussion phase!

[1] At least many physicists agree.