Spherical Tree-Sliced Wasserstein Distance

Q2. The tree structure is supposed to capture 'richer' topological information per the authors' claim. How does that translate to practical results? Have the authors explored different hyperparameters to confirm that the superior performance in these setups is due to the tree component of the method? Traditional trees in euclidean spaces often have a hierarchical structure; here, it appears that our design choice is not hierarchical? If so, then what are the concrete benefits?

Answer. Let us answer these questions by discussing more about the motivation for this paper. It arose from a simple yet intriguing idea: In the framework of Sliced Wasserstein (SW), a probability distribution on $\mathbb{R}^d$ is pushed forward onto a line. This raises the question: what does the resulting distribution reveal about the original one? It is evident that distinct distributions, when projected onto the same line, can become indistinguishable.

The situation is similar in the spherical setting. Given, for example, vertical SW, where each slice corresponds to a great semicircle, after rotating a spherical distribution around the diameter of a slice, the projected distribution of on that slice remains unchanged. This means that distinct distributions can become indistinguishable when projected onto the same slice. However, with spherical trees that include more than one great semicircle, the splitting map comes into play. It allows for differentiating distributions that are otherwise indistinguishable under rotation. As a result, two distributions that vertical SW cannot distinguish due to rotational symmetry can now be separated using the spherical tree structure in STSW.

In summary, with the same number of edges (as vertical SW), and thus the same computational cost, spherical trees in STSW provide a significantly deeper understanding of probability distributions compared to individual edges as in vertical SW. While more complex and hierarchical tree structures could be explored with the potential for improved performance, we opted for the simple structure described in the paper to ensure efficient implementation.

A natural question arises: if a better representation space is desired, why not replace one-dimensional manifold with higher-dimensional manifolds? The answer lies in computational feasibility. Optimal Transport in $\mathbb{R}^d$ for $d>1$ is computationally prohibitive due to the lack of a closed-form solution. In contrast, both vertical SW and STSW offer efficient closed-form expressions, making them more practical.

We believe this intuitive interpretation for STSW adequately addresses the reviewer's concerns.

We sincerely thank the reviewer for the valuable feedback. The typos highlighted by the reviewer have been corrected in the revision of our paper. If our responses satisfactorily address the concerns raised, we kindly hope the reviewer will consider increasing the score of our paper.

We appreciate the reviewer’s feedback and have provided the following responses to address the concerns raised about our paper. Below, we summarize the weaknesses and questions highlighted by the reviewer and provide our answers accordingly.

W1. Sampling: In the algorithm, the authors propose sampling uniformly from R^{d+1} and then normalizing to get points on S^d, which does not produce a uniform distribution on the sphere. Would this induce a bias (and implications)?

Answer. This is a typo in our manuscript, and we thank the reviewer to point it out. The correct term is standard normal distribution on $\mathbb{R}^{d+1}$, not uniform distribution on $\mathbb{R}^{d+1}$. We have revised this in our paper. It is worth noting that, except for the origin, the pushforward of the standard normal distribution on $\mathbb{R}^{d+1}$ via normalization map results in a uniform distribution on $\mathbb{S}^d$.

W2 + Q3. Ablation: The paper would be strengthened if there are more insights provided via ablations on different design choices (i.e., rays, trees). How does the current tree structure help capture the data better than existing methods? Are there limitations, theoretical issues, or numerical instability associated with different components of the method (i.e., S3W [3] has the north pole issue). Can the splitting maps be learned? etc.

Are there relationships to the OT distance on the spheres?

Answer. We provided some ablation studies of STSW in Appendix B. Given that the paper focuses on the construction of spherical trees and the corresponding Radon Transform, and the content is already comprehensive, we have decided to leave the analysis of tree structures, and statistical aspects of STSW for future work.

It is worth noting that analyzing these aspects of STSW is challenging due to the introduction of splitting maps. This component is unique to Tree-Sliced Wasserstein variants, distinguishing them from Sliced Wasserstein variants. We are actively working on analyzing splitting maps, and it appears to be a highly promising research direction.

"Can the splitting maps be learned?": In our paper, the splitting map is designed based on the distance from a point to the edges of spherical trees. Intuitively, the proportion of mass at a given point is proportional to its distance from the tree's edges. Splitting maps could potentially be made learnable by parameterizing them as a multi-layer perceptron (MLP) with a softmax layer at the end, allowing for end-to-end training with the model. However, this is a preliminary idea, and we have not empirically verified it yet. We leave this exciting idea for future work.

W3 + Q1. Runtime and Complexity: It would be nice to have an explicit discussion of the computational/memory complexity (this is aside from the information provided in Appendix B).

What makes STSW run faster than S3W [3], and in some cases, SW?

Answer. To demonstrate the scalability of the method, we present the computational complexity of calculating STSW using the closed-form approximation described in Eq. (19). The complexity is $\mathcal{O}(LNlogN + LkNd)$, which is theoretically equivalent to that of many sliced methods, such as SW. In practice, as shown in the Experimental Results section, STSW achieves favorable runtime performance. Notably, the closed-form approximation is a crucial factor contributing to the efficiency of our method.

W4. Experiments: This may be a minor point, but setup and hyperparameters could be better documented for all methods. In addition, there is no comparison with Vertical SW in appropriate setups. For generative experiments, there are no samples, of quantitative measures of the quality of images (i.e. the FID score).

Answer. Answering this question from the reviewer involves preparing code and conducting several experiments. We will provide a detailed response within a few days.

We appreciate the reviewer’s feedback and have provided the following responses to address the concerns raised about our paper. Below, we summarize the weaknesses and questions highlighted by the reviewer and provide our answers accordingly.

W1. The proposed metric is limited to hyperspheres. The paper's contributions appear to be incremental, primarily combining previously established concepts (tree systems and Sliced Wasserstein distances) and extending them to the sphere setting.

Answer. We believe there is a misunderstanding of the contributions of our paper. Please allow us to clear this misunderstanding by clarifying the novelty of our proposed Spherical Tree-Sliced Wasserstein. Our proposed metric is specifically designed for distributions on hyperspheres. The use of spherical distributions is widespread and is discussed in detail in the Introduction section of the paper. Existing studies on spherical variants of sliced Optimal Transport also primarily focus on data on hyperspheres, such as in [1], [2], [3], and others.

In our view, although the paper combines established concepts like tree systems and Sliced Wasserstein distances, the combination is far from straightforward. For instance:

• The tree systems in [4] and our proposed spherical trees are constructed differently, and their corresponding Radon Transforms also differ significantly.
• The splitting map in our work is more comprehensively designed than those in [4], as it considers both the positional information of points and lines, rather than just lines. It leads to a theorem for injectivity of our Radon Transform (Theorem 4.3), which requires a non-trivial proof.

For these reasons, we firmly believe our work demonstrates sufficient novelty for the conference.

W2. One of the claimed contributions is a bit misleading. Specifically, the authors claim to derive a closed-form expression; however, its computation still relies on approximations: first, due to the need for sampling trees, and second, by considering only discrete distributions in the explanation of how it is computed. This should be clarified, as the current claim gives the impression that the distance can be computed exactly due to the closed-form expression.

Answer. We agree with the reviewer that while Eq. (19) provides a closed-form expression for the Wasserstein distance in a tree metric, it does not yield a closed-form expression for STSW, but rather an approximation. Nonetheless, this approximation facilitates efficient implementation, and empirical results demonstrate that STSW performs effectively with this approach.

We have revised our paper according to the reviewer's feedback.

W3. Line 65 states that the use of tree systems enhances the capture of topological information. However, it is not so clear to me why is this the case. An experiment demonstrating this advantage would be useful.

Answer. Please allow us to clarify the motivation for our paper. It arose from a simple yet intriguing idea: In the framework of Sliced Wasserstein (SW), a probability distribution on $\mathbb{R}^d$ is pushed forward onto a line. This raises the question: what does the resulting distribution reveal about the original one? It is evident that distinct distributions, when projected onto the same line, can become indistinguishable.

The situation is similar in the spherical setting. Given, for example, vertical SW, where each slice corresponds to a great semicircle, after rotating a spherical distribution around the diameter of a slice, the projected distribution of on that slice remains unchanged. This means that distinct distributions can become indistinguishable when projected onto the same slice. However, with spherical trees that include more than one great semicircle, the splitting map comes into play. It allows for differentiating distributions that are otherwise indistinguishable under rotation. As a result, two distributions that vertical SW cannot distinguish due to rotational symmetry can now be separated using the spherical tree structure in STSW.

In summary, with the same number of edges (as vertical SW), and thus the same computational cost, spherical trees in STSW provide a significantly deeper understanding of probability distributions compared to individual edges as in vertical SW.

A natural question arises: if a better representation space is desired, why not replace one-dimensional manifold with higher-dimensional manifolds? The answer lies in computational feasibility. Optimal Transport in $\mathbb{R}^d$ for $d>1$ is computationally prohibitive due to the lack of a closed-form solution. In contrast, both vertical SW and STSW offer efficient closed-form expressions, making them more practical.

We believe this explanation adequately addresses the reviewer's concerns.

Q1. Impact uniformity loss: How should we understand $STSW(z^{A}, v)$ in eq. 20? $z^{A}$ is a single point, so I assume we are considering a Dirac distribution at $z^{A}$. Theoretically, the distance from a Dirac distribution to a uniform distribution in the sphere is constant regardless of where the Dirac distribution is placed, due to invariance to rotations, right? Why does this term favour uniformity if it is a constant term? Or am I misunderstanding something?

Answer. $z^{A}, z^B \in \mathbb{R}^{n \times (d + 1)}$ are the representations from the network projected on the hypersphere of two augmented versions of the same image. Thus, $z^A$ is not a single point. The self-supervised loss in Eq. (20) is proposed in [8] (See Eq. (86)), and we closely follow their setting for this experiment.

Q2. Radon Transform measure preservation: Why does the proposed Radon transformation in eq. 8 transform a probability distribution $\mu$ defined on $\mathbb{S}^d$ into a probability distribution defined on $\mathbb{S}^d$ This is mentioned in lines 332-333, but in line 268 it says that $\|| \mathcal{R}^\alpha_\mathcal{T}f \||_{\mathcal{T}} \le \|| f \||_1$. So it does not immediately follow that the Radon transform preserves the measure.

Answer. In the case where $f$ is a probability distribution on $\mathbb{S}^d$, $\mathcal{R}^\alpha_\mathcal{T}f$ is a distribution of $\mathcal{T}$. This follows directly from the proof provided in Appendix A.1. We recall the proof as follows: Since $f$ is non-negative on $\mathbb{S}^d$, $\mathcal{R}^\alpha_\mathcal{T}f$ is also non-negative on $\mathcal{T}$. Moreover,

$\|\|\mathcal{R}^\alpha_{\mathcal{T}}f\|\|_{\mathcal{T}}$

$= \sum_{i=1}^k \int_{0}^\pi \left|\mathcal{R}^\alpha_{\mathcal{T}}f(t, r^x_{y_i}) \right| \, dt$

$= \sum_{i=1}^k \int_{0}^\pi \left|\int_{\mathbb{S}^d} f(y) \cdot \alpha(y, \mathcal{T})_i \cdot \delta(t - \operatorname{arccos}\left<x,y \right>) ~ dy \right| ~ dt$

$= \sum_{i=1}^k \int_{0}^\pi \left(\int_{\mathbb{S}^d} |f(y)| \cdot \alpha(y, \mathcal{T})_i \cdot \delta(t - \operatorname{arccos}\left<x,y \right>) ~ dy \right) ~ dt$

$= \sum_{i=1}^k \int_{\mathbb{S}^d} \left(\int_{0}^\pi |f(y)| \cdot \alpha(y, \mathcal{T})_i \cdot \delta(t - \operatorname{arccos}\left<x,y \right>) ~ dy \right) ~ dt$

$= \sum_{i=1}^k \int_{\mathbb{S}^d} |f(y)| \cdot \alpha(y, \mathcal{T})\_i \cdot \left( \int_{0}^\pi \delta(t - \operatorname{arccos}\left<x,y \right>) ~ dt \right) ~ dy$

$= \sum_{i=1}^k \int_{\mathbb{S}^d} |f(y)| \cdot \alpha(y, \mathcal{T})_i ~ dy$

$= \int_{\mathbb{S}^d} |f(y)| \cdot \left (\sum_{i=1}^k \alpha(y, \mathcal{T})_i \right) ~ dy$

$= \int_{\mathbb{S}^d} |f(y)| ~ dy$

$= \||f\||_1 =1.$

Thus, $\mathcal{R}^\alpha_\mathcal{T}f$ is a probability distribution on $\mathcal{T}$.

Q3. STSW Computation on continuous measures: In section 5 you explain how to compute STSW in practice, but it is assumed that the probability distributions are discrete. Is it possible to get a closed form analogous to that in eq. 19 for non-discrete distributions?

Answer. Equation (19) is derived directly from the closed-form expression presented in [6]. For a general probability distribution, a closed-form expression can be obtained by replacing the summation with integration, as demonstrated in [7]. This approach is analogous to the well-known closed-form expression for the 1-dimensional Wasserstein distance: formulas for general distributions involve integrations, while those for discrete distributions use summation. In practice, implementations for discrete distributions rely on fundamental operations such as matrix multiplication, sorting, and similar techniques.

In applications, we typically work with discrete probability distributions. This is why we focus on discrete probabilities in the paper.

Q4. Injectivity of the radon transform: In Theorem 4.3 it is proved that if the splitting map is invariant, then the spherical Radon transform is invariant. What would be the consequences of using a non-injective spherical Radon transform? What structure might be missed?

Answer. This is a significant contribution of our paper, as the injectivity of a Radon transform variant is often a crucial requirement. It determines whether the derived metric (such as STSW) qualifies as a true metric or remains a pseudo-metric. Without injectivity, the Radon transform could lead to a pseudo-metric, allowing the possibility of two distinct probability distributions having a distance of zero. Consequently, using a pseudo-metric in applications could result in unstable performance.

The study of injectivity in Radon Transform variants has been extensively explored in numerous studies, including [1], [2], [3], [4], [5], and others.

Reference.

[1] Tran, Huy, et al. "Stereographic spherical sliced wasserstein distances." arXiv preprint arXiv:2402.02345 (2024).

[2] Quellmalz, Michael, Robert Beinert, and Gabriele Steidl. "Sliced optimal transport on the sphere." Inverse Problems 39.10 (2023):105005.

[3] Quellmalz, Michael, Léo Buecher, and Gabriele Steidl. "Parallelly sliced optimal transport on spheres and on the rotation group." Journal of Mathematical Imaging and Vision (2024): 1-26.

[4] Tran, Viet-Hoang, et al. "Tree-Sliced Wasserstein Distance on a System of Lines." arXiv preprint arXiv:2406.13725 (2024).

[5] Kolouri, Soheil, et al. "Generalized sliced wasserstein distances." Advances in neural information processing systems 32 (2019).

[6] Le, Tam, et al. "Tree-sliced variants of Wasserstein distances." Advances in neural information processing systems 32 (2019).

[7] Le, Tam, et al. "Sobolev transport: A scalable metric for probability measures with graph metrics." International Conference on Artificial Intelligence and Statistics. PMLR, 2022.

[8] Bonet, Clément, et al. "Spherical sliced-wasserstein." arXiv preprint arXiv:2206.08780 (2022).

We sincerely thank the reviewer for the valuable feedback. The typos highlighted by the reviewer have been corrected in our paper, and we plan to update the manuscript within a few days.

If our responses satisfactorily address all the concerns raised, we kindly hope the reviewer will consider increasing the score of our paper.

We appreciate the reviewer’s feedback and have provided the following responses to address the concerns raised about our paper. Below, we summarize the weaknesses and questions highlighted by the reviewer and provide our answers accordingly.

W1. The approach builds on previous work by Bonet et al. (2022) and Tran et al. (2024a), in the sense that uses the same hight-level ideas. However, while the research incrementally follows the line of previous studies by Bonet et al. (2022), Tran et al. (2024a), and Tran et al. (2024b), it offers meaningful advancements by developing a metric specifically adapted for spherical data analysis. Also, the experiments closely follows experiments previously presnted in papers as Bonet et al. (2022).

Answer. This is an interesting point and we are enthusiastic about delving deeper into it. Roughly speaking, the tree-sliced framework that is adapted in our paper is built on two key insights:

• Local perspective: Each edge in a spherical tree is treated similarly to an one-dimensional slice in existing Spherical Sliced Wasserstein (SSW) frameworks. Splitting maps determine how the mass at each point is distributed across the lines, and then the projection of these mass portions onto the lines is processed in the same way as in SW.
• Global Perspective: Spherical tree structures and splitting maps establish connections between the lines, creating a cohesive system. This introduces a novel aspect compared to SSW, enabling interaction and integration among the edges in a spherical tree. The Wasserstein distance can now be computed on this space with a closed-form expression, analogous to how one-dimensional manifolds are treated in SSW.

It is important to note that while these ideas might seem straightforward, their development is non-trivial. A key challenge lies in ensuring the injectivity of the corresponding Radon Transform, which is critical in determining whether the proposed metric is a true metric or merely a pseudo-metric. We have addressed this issue by providing a rigorous proof in the paper.

In the experimental sections, we follow the standard experimental setups described in [1] and [2], which are widely used benchmarks for assessing the performance of Spherical Sliced Wasserstein methods.

Q1. Besides the experimental comparisons of the new STSW with SW, SSW, and S3W variants, are there any analytic comparisons among them? Which are the differents in the topologies defined by these different approach?

Answer. Given that the paper focuses on the construction of spherical trees and the corresponding Radon Transform, and the content is already comprehensive, we have decided to leave the analytical and statistical aspects of STSW for future work.

It is worth noting that analyzing these aspects of STSW is challenging due to the introduction of splitting maps. This component is unique to Tree-Sliced Wasserstein variants, distinguishing them from Sliced Wasserstein variants. We are actively working on analyzing splitting maps, and it appears to be a highly promising research direction.

Reference.

[1] Bonet, Clément, et al. "Spherical sliced-wasserstein." arXiv preprint arXiv:2206.08780 (2022).

[2] Tran, Huy, et al. "Stereographic spherical sliced wasserstein distances." arXiv preprint arXiv:2402.02345 (2024).

We sincerely thank the reviewer for the valuable feedback. If our responses adequately address all the concerns raised, we kindly hope the reviewer will consider raising the score of our paper.

We would like to thank the reviewer again for your thoughtful reviews and valuable feedback.

We would appreciate it if you could let us know if our responses have addressed your concerns and whether you still have any other questions about our rebuttal. We would be happy to do any follow-up discussion or address any additional comments.

If you agree that our responses to your reviews have addressed the concerns you listed, we kindly ask that you consider whether raising your score would more accurately reflect your updated evaluation of our paper. Thank you again for your time and thoughtful comments!

I thank the authors for clarifying some of my queries. Particularly, I agree with their comment "It is important to note that while these ideas might seem straightforward, their development is non-trivial. A key challenge lies in ensuring the injectivity of the corresponding Radon Transform, which is critical in determining whether the proposed metric is a true metric or merely a pseudo-metric. We have addressed this issue by providing a rigorous proof in the paper." I will increase my score. I have no further comments.

We appreciate the reviewer’s feedback and have provided the following responses to address the concerns raised about our paper. Below, we summarize the weaknesses and questions highlighted by the reviewer and provide our answers accordingly.

W1. While STSW aims to be efficient, the spherical Radon transform and tree-slicing require considerable computation, especially as the number of edges or the dimension of the hypersphere increases. This could limit scalability for very high-dimensional or densely-sampled spherical data, impacting runtime in large-scale applications. Although I understand that the authors have provided extensive runtime comparisons, I would like to see the scalability of the method on higher-dimensional tasks beyond CIFAR, MNIST, and similar datasets.

Answer. To demonstrate the scalability of the method, we present the computational complexity of calculating STSW using the closed-form approximation described in Eq. (19). The complexity is $\mathcal{O}(LNlogN + LkNd)$, which is theoretically equivalent to that of many sliced methods, such as SW. In practice, as evidenced in the Experimental Results section, STSW exhibits favorable runtime performance. Notably, the closed-form approximation is a crucial factor contributing to the efficiency of our method.

W2+Q1. The effectiveness of STSW relies on the quality of sampled spherical trees, which introduces variability in metric accuracy. If the sampling process fails to capture diverse spherical structures adequately, STSW’s results might be inconsistent, especially in complex distributions where more refined tree structures are necessary. I would like to see which strategies (e.g., Markov Chains, Random Paths, etc.) could be applied here to sample more informative slices.

I was wondering if incorporating a Markov chain over the distributions of the slices, instead of using a uniform distribution, could help in generating more informative tree-slices.

Answer. We appreciate the reviewer for suggesting an approach to enhance the STSW method. Exploring more complex distributions for the slices, rather than relying solely on a uniform distribution, is indeed a promising direction. Similar strategies have been adopted in some studies on the Sliced Wasserstein method, such as [1], [2], and others.

For STSW, employing more complex distributions for the slices, such as integrating a Markov chain over the slice distributions, is anticipated to enhance the method's effectiveness. Sampling the roots and edges of spherical trees can be adapted to the data, potentially through a learnable sampling process.

Given that the paper focuses on the construction of spherical trees and the corresponding Radon Transform, and the content is already comprehensive, we have decided to leave a more in-depth exploration of tree sampling methods for future work.

Q3. I am interested in understanding the topology of the tree corresponding to the most informative slice in spherical trees and compare its effectiveness to the most informative slice in SSW, and essentially comparing them to the most informative slice in SWD. (By the most informative slice I mean Max-slice). This can be done on a chosen benchmark.

Answer. We have conducted an experiment for most informative slice method including MAX_STSW, MAX_SSW and MAX_SW on gradient flow task aimed at learning target distribution of 12 vMFs. We present in table below the results after training for 1000 epochs with learning rate $LR=0.01$. Each experiment is repeated 10 times.

Table 1: Learning target distribution 12 vFMs, LR=0.01, 1000 epochs, averaged over 10 runs

We have also included a figure in the paper to visualize this experiment.

Reference.

[1] Deshpande, Ishan, et al. "Max-sliced wasserstein distance and its use for gans." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.

[2] Nguyen, Khai, et al. "Distributional sliced-Wasserstein and applications to generative modeling." arXiv preprint arXiv:2002.07367 (2020).

We sincerely thank the reviewer for the valuable feedback. If our responses adequately address all the concerns raised, we kindly hope the reviewer will consider raising the score of our paper.

We would like to thank the reviewer again for your thoughtful reviews and valuable feedback.

We would appreciate it if you could let us know if our responses have addressed your concerns and whether you still have any other questions about our rebuttal. We would be happy to do any follow-up discussion or address any additional comments.

If you agree that our responses to your reviews have addressed the concerns you listed, we kindly ask that you consider whether raising your score would more accurately reflect your updated evaluation of our paper. Thank you again for your time and thoughtful comments!

Thanks for your response. We appreciate your endorsement and your acknowledgment of our contributions.

We appreciate the reviewer’s feedback and have provided the following responses to address the concerns raised about our paper.

Q1. 419: "STSW outperforms the baselines in all metrics and achieves faster convergence." Why is that? Is this result theoretically predictable?

Answer. This is an empirical result for the Gradient Flow task. As shown in Table 1 and Figure 10, STSW outperforms the baselines in all metrics and achieves faster convergence. Providing theoretical explanation for this observation requires a deeper exploration of the analytical and statistical dimensions of STSW, which we are currently pursuing as part of our future work on STSW. (See Q3 also.)

Q2. 466: "We also conduct experiments with to visualize learned representations." Why don't we directly project the learned features in the original image space to a sphere, rather than redoing the experiments on the sphere?

Answer. We closely adhere to the experimental setups outlined in [1] and [2], which are commonly used as benchmarks for evaluating the performance of Spherical Sliced Wasserstein methods. Directly projecting the learned features onto a sphere may lead to some loss of information. Moreover, [1] and [2] provide the visualization of the projections on $\mathbb{S}^{2}$, so we include similar visualizations for better comparison.

Q3. 443: The variances from STSW are quite small. What's the explanation of that? Why doing tree-slicing on spheres is more robust (against different sampling?)?

Answer. Given that the paper focuses on the construction of spherical trees and the corresponding Radon Transform, and the content is already comprehensive, we have decided to leave the analytical and statistical aspects of STSW for future work.

It is worth noting that analyzing these two aspects of STSW is challenging due to the introduction of splitting maps. This component is unique to Tree-Sliced Wasserstein variants, distinguishing them from Sliced Wasserstein variants. We are actively working on analyzing splitting maps, and it appears to be a highly promising research direction.

Q4. We have seen spherical trees in the following article, although it's for a different application and the construction of the trees is different as well. Is there any connection between this work and that?

Meng, Yu, Yunyi Zhang, Jiaxin Huang, Yu Zhang, Chao Zhang, and Jiawei Han. "Hierarchical topic mining via joint spherical tree and text embedding." In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1908-1917. 2020.

Answer. We appreciate the reviewer for bringing the referenced paper to our attention. While both that paper and ours use the term "spherical tree," the meanings are distinct. In our view, there is likely no connection between the two works.

Reference.

[1] Bonet, Clément, et al. "Spherical sliced-wasserstein." arXiv preprint arXiv:2206.08780 (2022).

[2] Tran, Huy, et al. "Stereographic spherical sliced wasserstein distances." arXiv preprint arXiv:2402.02345 (2024).

Once again, we sincerely thank the reviewer for their feedback. Please let us know if there are any additional concerns or questions from the reviewer regarding our paper.

W3. Another point is that the authors attributes the better performances to "the ability to capture topological information of integration domain" of STSW but they didn't show a direct connection in the paper.

Answer. Let us discuss the motivation behind our paper. It comes from a simple yet intriguing idea: In the framework of Sliced Wasserstein (SW), a probability distribution on $\mathbb{R}^d$ is pushed forward onto a line. This raises the question: what does the resulting distribution reveal about the original one? It is evident that distinct distributions, when projected onto the same line, can become indistinguishable.

The situation is similar in the spherical setting. Given, for example, vertical SW, where each slice corresponds to a great semicircle, after rotating a spherical distribution around the diameter of a slice, the projected distribution of on that slice remains unchanged. This means that distinct distributions can become indistinguishable when projected onto the same slice. However, with spherical trees that include more than one great semicircle, the splitting map comes into play. It allows for differentiating distributions that are otherwise indistinguishable under rotation. As a result, two distributions that vertical SW cannot distinguish due to rotational symmetry can now be separated using the spherical tree structure in STSW.

In summary, with the same number of edges (as vertical SW), and thus the same computational cost, spherical trees in STSW provide a significantly deeper understanding of probability distributions compared to individual edges as in vertical SW.

A natural question arises: if a better representation space is desired, why not replace one-dimensional manifold with higher-dimensional manifolds? The answer lies in computational feasibility. Optimal Transport in $\mathbb{R}^d$ for $d>1$ is computationally prohibitive due to the lack of a closed-form solution. In contrast, both vertical SW and STSW offer efficient closed-form expressions, making them more practical.

We believe this explanation sufficiently answers the question raised. It is also the same concern we included in the General Response.

W4. The results from the CIFAR dataset are not that impressive and the visualization didn't help either in explaining them.

Answer. For self-supervised learning task, we recall two properties for evaluating the representation quality, which are Alignment and Uniformity (proposed in [1]):

• Alignment: Ensures that similar samples are assigned similar features.
• Uniformity: Promotes an even distribution of features across the hypersphere.

In Table 2, we consider the reported accuracy to be a significant improvement. For example, focusing on the accuracy of encoded features (Acc. E), S3W-related methods [2] achieve the best result of $80.08\\%$, with the second-best result from other baselines being SimCLR at $79.97\\%$. In contrast, our method achieves an even higher accuracy of $80.53\\%$.

Notably, STSW demonstrates competitive efficiency, achieving the second-best runtime at 9.54 seconds per epoch, closely following SimCLR, which achieves the best runtime at 9.34 seconds per epoch.

In Figure 12, we present the learned representations of various methods. We observe that STSW demonstrates a balanced combination of these two properties.

Reference.

[1] Wang, Tongzhou, and Phillip Isola. "Understanding contrastive representation learning through alignment and uniformity on the hypersphere." International conference on machine learning. PMLR, 2020.

[2] Huy Tran et al., Stereographic Spherical Sliced Wasserstein Distances. ICML 2024.

We sincerely thank the reviewer for the valuable feedback. If our responses adequately address all the concerns raised, we kindly hope the reviewer will consider raising the score of our paper.

We would like to thank the reviewer again for your thoughtful reviews and valuable feedback.

We would appreciate it if you could let us know if our responses have addressed your concerns and whether you still have any other questions about our rebuttal.

We would be happy to do any follow-up discussion or address any additional comments.