topological registration losses is not demonstrated

See Global Rebuttal and companion pdf (Figure 2(b) and Table 1), where we showcased our methodology on a topological registration loss on a real-world dataset of single cells. While we do observe oscillations, we still note a global decrease of the loss. As for the correlation score, the improvement is very marginal, which we believe is due to the fact that correlation score only measures the correlation between latent space angles and ground truth. Thus, while the optimized latent space does exhibit a better delineated circular shape with the registration loss (see Figure 2(b)), the size of that circle is prescribed by the coordinates of the target PD point $[-3.5, 3.5]$ (whereas this size is made as large as possible with the augmentation loss), which in turn does not change the latent space angles very much (as well as the correlation score overall) compared to the augmentation loss.

The experiments on convergence rate are only designed for simple topological situations. When the topological complexity increases, the optimization becomes unstable. Will the proposed method introduce additional instability to the optimization? In other words, could a convergence theorem for the proposed method be derived?

Thank you for the suggestion. Proving a convergence theorem is indeed one of the future directions we aim at pursuing. We believe that such convergence results should be accessible provided that existing results in this vein (such as, e.g, [Optimizing persistent homology based functions, Carrière et al., ICML, 2021]) can be thought of as particular instances of our setting when $\sigma \to 0$. Hence, convergence should also occur for $\sigma$ and learning rates that are sufficiently small, in order to ensure that they do not interfere too much with the different terms involved by the high topological complexity.

Guaranteeing the convergence for fixed bandwidth $\sigma$ and learning rate $\lambda$ in the greatest generality seems too demanding: we designed in Global Rebuttal and the companion pdf some cases where convergence seems to fail numerically because $\sigma$ is much too large with respect to the diameter of the point cloud. As commented in Global Rebuttal, a good heuristic in our experiments is to take $\sigma$ as a fraction of the median distance in the point cloud.
Therefore, an appealing generalization suggested by reviewer v7wD would be to use in practice an adaptive / locally varying $\sigma$ that would in some sense, adapt to small vs large topological features. Note that one should do it in a way such that $K(x,y)$ remains positive semi-definite for any $x,y$ though, so that the theory of diffeomorphic interpolations presented in Section 2.2 still applies.

1. What is the intuition behind the proof of Theorem 2?

Our goal with this result was to quantify the smoothness of our proposed diffeomorphic interpolation (as characterized with its Lipschitz constant) based on the kernel it is computed from. In the case of the Gaussian kernel, we found that the Lipschitz constant depends on the kernel bandwidth $\sigma$: indeed, the more spread the Gaussian is, the more critical points (identified during the computation of PH) can influence other data points potentially far from them, introducing unwanted distortions and larger Lipschitz constant. Similarly, if the condition number is large, inverting the kernel matrix might introduce instabilities in the $\alpha_i$ in Equation (4), and thus a larger Lipschitz constant as well. Finally, the total persistence also appears, as the more PD points one has to optimize, the more critical pairs will appear, and thus the more constrained the minimization problem in section 2.2 is, leading to more complex diffeomorphic interpolation solutions with larger Lipschitz constants.

2. What do the theoretical guarantees tell us in practice? (…) evolution of loss with discrete step-size? (…) what makes it much harder?

We agree that the theoretical guarantees remain of limited use in practice: they only tell us that our diffeomorphic interpolations also provide meaningful descent directions for the loss; and thus that for a sufficiently small step-size, the loss must decrease. Empirically, we showcase that overall, the loss decreases substantially faster using our method. Proving this formally in some reasonable settings (continuous-time, etc.) remains quite hard though, for various reasons:

• the losses considered are typically not convex (with respect to the input point cloud $X$), and thus it is hard to quantify how large the learning rate can be taken and how much the loss decreases after a discrete-time step,
• Our $\tilde{v}$ that dictates the dynamic of the flow is not a gradient (at least for Euclidean geometry), so we cannot apply faithfully the gradient flow/descent literature to get theoretical guarantees,
• The dependence of the Vietoris--Rips filtration on the point cloud $X$ is cumbersome. We can derive theoretical guarantees as long as the ordering of the pairwise distances $\|x_i - x_j\|$ remains unchanged (i.e., as long as the points are moved by less than $\min_{ij,kl} |\|x_i - x_j\| - \|x_k - x_l\| |$), but this is fairly restrictive.

It is clear that studying this model is of interest, but we believe that it may be postponed for future work.

3. and 4. How did you choose the bandwidth in practice? (…) heuristic ? (…) vary sigma locally ? (…) different choices of sigma ?

We chose $\sigma$ with no parameter tuning (see Global Rebuttal and companion pdf, which actually show that the results of our method in Figure 3 can actually be improved if we pick $\sigma = 0.3$ instead of $0.1$); the heuristic being that $\sigma$ should be a fraction of the median distance between input points (this heuristic is common in calibration of Gaussian kernel in other statistical scenarios).

It may be possible to consider a space-dependent $\sigma$ or other variant: as long as $(x,y) \mapsto K(x,y)$ remains valued in positive semidefinite matrices, the theory presented in Section 2.2. adapts faithfully (see also line 154 in the main paper).

We included in Global Rebuttal and companion pdf (Figures 1 and 2(a)) some experiments for varying $\sigma$, showcasing the robustness of our approach w.r.t. $\sigma$.

5. Oineus loss larger

Good catch, thanks! We forgot a normalization factor in the way the oineus loss is displayed. Fortunately, as our stopping criterion is that a loss of exactly $0$ should be reached (see line 248 in the main paper), this does not change the number of iterations (e.g., 49 iterations for oineus alone) nor the running times, and thus the conclusions of this proof-of-concept experiment remain unchanged. We will correct this in the next version.

6. In Figure 3, how did you pick the hyperparameters? (…) It might be the case that different hyperparameter regimes are optimal.

We actually pick $\lambda = 0.1$ for every experiments, with no specific tuning. Given that our approach is directly extrapolating the vanilla topological gradient, we believe that picking the same learning rate for both models is the natural choice.

Figure 9, (a) 3rd from left (…) purple points? (b) 4th (…) loss spikes for the diffeo algorithm be explained?

(a) For 3rd Figure, transparent blue points simply represent the initial point cloud as a reference. Orange points represent the final state of the point cloud.

(b) We report the loss on the subsample batch (because evaluating the loss on the whole point cloud is computationally prohibitive, see line 227). Because of the stochastic nature of our approach, it may happen that this estimated loss is, on a few occasions, quite high because the subsample does not reflect the expected topological structure.

This does not happen with vanilla gradient descent for a simple reason: the initial point cloud is broadly uniform, so are the subsamples (with high probability) yielding a loss around $-0.03$, and because vanilla gradient descent fails to perform any significant update of the point cloud, this situation remains through iterations. In contrast, because our diffeomorphic interpolations do change the point cloud, they reach configurations where some subsamples may yield a high loss, though the loss decreases overall.

In a nutshell, these spikes are not related to the optimization technique involved (if the vanilla gradient descent was initialized with, say, the epoch 600 reached by our approach based on diffeomorphisms, one would observe similar spikes), but to the “topological variance” in the subsample for a given configuration, the point being that vanilla gradient descent does not reach such configurations.

Dear reviewer v7wD,

Thank you for your review. The authors have tried to address your concerns in the rebuttal. Please carefully read their rebuttal, and let them know what you think, and whether there is any more clarifications you require. Note that author-reviewer discussion ends on August 13th. Thanks! the AC

(Q1) color swap in Figures 6 and 7

This was accidental. Thank you for catching this error!

(Q2) elaborate on (…) black-box AE models, (…) compare to other topological optimisation techniques

Sorry: we made a misleading mistake by accidentally naming “Vanilla” the competitor of “Diffeo” (ours) in the Table in page 9. Here, “Vanilla” is supposed to mean no topological optimization on the latent space (like “Vanilla VAE”, not “Vanilla topological gradient descent”). We will change this in the revised version and are deeply sorry for the confusion it yielded.

We do not compare our method with existing topological optimization methods (Oineus, Vanilla, etc.) in this experiment. That is for a good reason: the experiment we propose can only be made using our method. Indeed, given a pre-trained VAE $D \circ E$, we design $\varphi : \mathbb{R}^{d’} \to \mathbb{R}^{d’}$ such that $\varphi \circ E$ induces a “better” latent representation (in terms of topological properties) by concatenating our diffeomorphic interpolations $\tilde{v}_k, k=1,\dots,T$.

In contrast, existing methods do not provide maps defined on the latent space. Their gradients are only defined on training observations. Given (say) a new latent representation $z$, there is no way to “correct it” using vanilla or oineus gradients.

Aside from computational efficiency, this is the biggest edge of our method over pre-existing topological optimization techniques: having access to maps defined on the whole space opens the way for new applications.

(Q3) ran experiment multiple times?

See Global Rebuttal and companion pdf, Table 1. As for the Pig result, the large margin is because the initial space (from the VAE) was particularly bad (topologically), see Figure 2(b).

(Q4) Figure 3 (…) expected end result of topological optimisation? The final shape (…) is very distorted, (…) how is that good?

The loss that we minimize in this PoC experiment is proportional to the topological information contained in the input point cloud. That is, we want to destroy the topological structure (here, the presence of a loop). A good output (low loss) is a point cloud with no underlying loop. The take-home messages of this experiment are:

• The vanilla gradient only updates a few points (sparse gradient) of the (small) subsample. This yields a low decrease in the loss: after $t=100$ steps, the point cloud has barely changed.
• In sharp contrast, our diffeomorphic interpolation updates the whole point cloud at each step. It yields a much faster optimization (showcased also in Figure 4), and the whole point cloud is deformed smoothly from the input one.
• In terms of raw topological loss, there is no “better” final output: both methods manage to eventually provide a point cloud with trivial topology. Nonetheless, having a smooth, invertible, deformation between the input and final objects can be useful and more interpretable; this is for instance leveraged in the black-box AE experiment.

(W1, real-world experiment) + (Q5) real-world uses of topological optimisation?

See Global Rebuttal and the companion pdf (Figure 2(b) and Table 1) for a real-world application on single-cell data. In a nutshell, topological optimization can be useful if you have a topological prior (e.g., "my model should exhibit a periodic / cyclic structure"). One can also consider the references given in lines 40--42 in the main paper for other possible applications of topological optimization.

(Q6) how big datasets can your approach handle?

Diffeomorphic interpolation updates computed from a subsample of the dataset modify the whole input point cloud, a sharp difference with the vanilla gradients that only update a fraction of the original point cloud at each step. Our PoC experiments illustrate this (in particular Figure 5 shows that our approach can seamlessly perform topological optimization on a point cloud with $\sim 36,000$ points, a scale completely out of reach of previous methods), and we confirmed this on our new real-world application, which involves Vietoris-Rips PH on $1,171$ single cells. While this is already hardly tractable for vanilla gradients, subsampling combined with our diffeomorphic interpolations allows to perform the task fastly and efficiently.

In terms of computational complexity, the overhead induced by our method compared to vanilla gradients are:

• inverting the kernel matrix $\mathbf{K}\in \mathbb{R}^{|I| \times |I|}$ where $I$ is the set of indices for which the vanilla gradient is non-zero. Since the vanilla gradient is usually sparse, $I$ is small and this matrix inversion is basically negligible.
• Computing $\tilde{v}$ on the whole input point cloud $X$, which scales linearly in the number of points.

(Q7) elaborate on (…) interpretability for black-box autoencoder?

The idea showcased in this experiment is the following: each dataset is a collection of images representing a rotating object, hence it is natural to assume that there must be a 2-dimensional underlying circle (topological prior). When training a VAE on such a dataset, we obtain an encoder $E$ and a decoder $D$ that enables the generation of new data. However, the latent space does not exhibit the expected circle structure (it has no reason to do so). We change the latent space representation using the diffeomorphism $\varphi$ optimized so that $(\varphi \circ E(x_i), i=1,\dots,n )$ satisfies our topological prior ($(x_i)_{i}$ is our training set). Since $\varphi$ is invertible, we can generate new observations using $D \circ \varphi^{-1}$.

This increases interpretability: after topological correction, the latent representation is organized on a circle, and angles in the latent space reflect angles between rotated images. You can now sample "an image rotated by 30°” (wrt some reference image) by simply taking an angle of 30° in the latent space.

(Q8) How robust (…) to (…) different samples

See Global Rebuttal and companion pdf, Figure 1.

Weaknesses 1. and 2. (Section 3 is notation intensive. // Proof of Proposition 3.1 can be moved to the appendix)

Thank you for the suggestion. We will alleviate Section 3, stating Proposition 3.2. more informally and deferring the complete, technical statement to the appendix. As far as the proof of Proposition 3.1 is concerned, as it is very short, we consider keeping it in the main paper as it shows in a simple way how $\tilde v$ interplays with the loss, which is helpful for developing intuition.

Weakness 3. (For the subsampling experiment, the comparison with vanilla topological gradient seems slightly unfair because there is no way for the network to update gradient over the whole space)

We agree. One could say that it is precisely the purpose of this experiment: since the vanilla topological gradients are not defined on the whole space (not even the whole input point cloud), and because of the inherent computational limitation of (Vietoris-Rips) persistent homology, it is expected that vanilla gradient descent will perform poorly on this task.

So yes, to some extent, the comparison is indeed unfair, but this is exactly the point we wanted to emphasize: extending gradients via diffeomorphic interpolations has a crucial impact in terms of computational efficiency, opening the way for new experiments involving topological optimization that are not currently accessible with vanilla gradient descent.

Minor (Should $X_t$ be $X_k$ ?)

Indeed, that’s a typo. Thank you for catching it!

Question : Can this method work for differentiable multiparameter vectorization methods?

Our approach may be generalized easily: as long as one is given a (sparse) gradient on a point cloud $X \in \mathbb{R}^{n \times d}$, one can build its diffeomorphic interpolation and use the resulting $\tilde{v}$ to update $X$ instead.

Therefore, any methods that returns such gradients can benefit from our approach. This includes recent pipelines involving differentiable multi-parameter PH such as D-GRIL [D-GRIL: End-to-end topological learning with 2-parameter persistence. Mukherjee et al. arXiv:2406.07100, 2024] and differentiable signed measures [Differentiability and convergence of filtration learning with multiparameter persistence. Scoccola et al. ICML, 2024]. In both methods, gradients can be obtained by computing free resolutions of multi-parameter persistence modules, and are thus also very sparse (similar to the 1-parameter case studied in our work) as they depend only on the multigraded Betti numbers of the module.