Toward a Unified Geometry Understanding : Riemannian Diffusion Framework for Graph Generation and Prediction

Dear Reviewer 8zMr,

We sincerely thank the reviewer for the detailed comments and insightful questions. Our response to your comments one by one as follows.

---

W1: The authors could potentially add some additional benchmarks [1] to increase the validity of their results.

Reply to W1: Thank you for pointing out the model and benchmarks we had previously overlooked. We will cite this work in the revised version and try to add the relevant metrics to the MOSES dataset, and incorporate the baseline model.

---

Q1: What is the computational cost of computing the gyrokernel?

Reply to Q1: Here, we analyze the complexity of our Riemannian gyrokernel mappings from theory and experiment.

Theory: Suppose that a general Fourier mapping $\phi_{\mathrm{gF}}(x)$ consists of m eigenfunctions and the number of representation vectors $V$ in different Riemannian spaces is $n$. First, the computation of each eigenfunction $\mathrm{gF}_{\boldsymbol{\omega},b,\lambda}^{\kappa}(\boldsymbol{x})$ involves only simple operations with the time complexity of $O(1)$. Thus ,the whole process of calculating $\overline{V}$ is O(mn). It is worth noting that since the settings of m and n are very small, typically mn<10. the process is therefore very fast.

Experiment: We report the time consumption in Table 1. Experimental results show that our mapping remains almost identical to the original method without significant time loss.

Table 1 time consumption on photo dataset in one epoch

Dear Reviewer aMiG,

We sincerely thank the reviewer for the detailed comments and insightful questions. Our response to your comments one by one as follows.

W1: The lack of visual representation of the generation process makes it impossible to vividly demonstrate how this self-guiding mechanism is generated. Q1: What is the denoising process like under the self-guided mechanism?

Reply to W1: We are very sorry that due to the latest rebottal notification, we are unable to provide the relevant figures. The self-guiding mechanism enables the model to generate features that are more similar to the original representation. In the later stage of denoising, the features of the same label will be closer. However, if the self-guiding mechanism is removed, the generated features will be scattered at various locations. We will add relevant visualization content in the revised version.

---

W2: The paper lacks a comparative discussion of existing Riemannian diffusion approaches[1][2]. It is important to explain why alternative Riemannian diffusion models were not adopted or compared. Q2: Why not use other Riemannian diffusion models?

Reply to W2: We clarify that one of the benefits for this design is to consider the efficiency issue of implementing diffusion directly on Riemannian geometry. Benefiting from the fact that the feature remains in the Euclidean space after the generalized Fourier mapping, this design does not require a complex execution of the diffusion process on the Riemannian space no matter whether to use Riemannian autoencoder. In addition, the Riemannian kernel method can indirectly enhance the representation of the latent space and thus improve the performance of diffusion model. This flexibility allows our architecture to be compatible with pre-trained graph latent diffusion models in the future without training a specialized Riemannian diffusion model from scratch.

---

W3: The self-guiding mechanism warrants a deeper analysis. For instance, it remains unclear whether the generated labels reflect underlying chemical properties of the molecules and, if so, how such relationships are captured or influenced by the guidance strategy. Q3: Can the author calculate the jaccard coefficients of the two categories after clustering based on their chemical properties?

Reply to W3: This is indeed a very valuable question. We conducted an analysis on the QM9 dataset to examine the relationship between clustering labels and the chemical attribute $\mu$. Since the chemical attribute values are continuous numerical data, we first sorted them based on their magnitudes and divided them into corresponding sets. We then computed the Jaccard index between these sets and those derived from the clustering labels. The average Jaccard score obtained was 0.1286, indicating that the self-guiding mechanism is only weakly related to the underlying chemical properties.

Although there exists a certain correlation between molecular structure and chemical properties, the representations used for generation may exhibit weaker correlations with chemical attributes, as they focus on different levels of information. We believe that the pseudo-labels produced by the self-guiding mechanism are primarily influenced by structural topology, which include the reconstructed representation information of nodes and edges.

Dear Reviewer c5kY,

We sincerely thank the reviewer for the detailed comments and insightful questions. Our response to your comments one by one as follows.

---

W1: The explanation of Figure 1(b) could be expanded. It remains unclear whether different feature levels correspond to distinct curvatures, dimensions, or structural complexities, and how these aspects interact. Q1: In Fig1(b),in what aspects is the complexity of multi-level data manifolds reflected? Are their curvatures the same? For example, for node classification or link prediction tasks, are node features and edge features completely different?

Reply to W1: On the one hand, features at different levels exhibit varying curvatures. We calculated the ricci curvature of different features as reported in Table 1. This indicates that the spatial curvatures of features across levels are distinct, highlighting the complexity of the data manifold. On the other hand, from the task perspective, different tasks encourage different manifolds. Node classification encourages nodes with the same label to learn similar features, whereas link prediction primarily emphasizes structural features. Thus, the differing task objectives further contribute to the complexity of the data manifold.

Table 1 curvature of multi-level embeddings

---

W2: I'm not very familiar with the ability of diffusion to capture data manifolds. The paper lacks a thorough discussion on how well the diffusion process captures data manifolds, particularly in the context of modeling multiple, potentially heterogeneous, geometric levels. Q2: Can the diffusion model effectively capture data manifolds? Are there any previous studies or theoretical derivations?

Reply to W2: To our knowledge, many studies[1][2][3] have demonstrated that diffusion models can effectively capture data manifolds. The key insight is that the score function learned by a diffusion model tends to point in the normal direction towards the data manifold. This property has been further leveraged to investigate intrinsic dimensions and other geometric characteristics of data manifolds.

[1] Diffusion Models Encode the Intrinsic Dimension of Data Manifolds.

[2] Adaptivity of Diffusion Models to Manifold Structures.

[3] Convergence of Diffusion Models Under the Manifold Hypothesis in High‑Dimensions.

---

W3: The computational cost of the proposed method is not addressed. A discussion of its time complexity would help assess the scalability and practicality of the approach. Q3: What is the computational complexity of the gyroscope kernel method mentioned in the paper?

Reply to W3: Here, we analyze the complexity of our Riemannian mappings from theory and experiment.

Theory: Suppose that a general Fourier mapping $\phi_{\mathrm{gF}}(x)$ consists of m eigenfunctions and the number of representation vectors $V$ in different Riemannian spaces is $n$. First, the computation of each eigenfunction $\mathrm{gF}_{\boldsymbol{\omega},b,\lambda}^{\kappa}(\boldsymbol{x})$ involves only simple operations with the time complexity of $O(1)$. Thus ,the whole process of calculating $\overline{V}$ is O(mn). It is worth noting that since the settings of m and n are very small, typically mn<10. the process is therefore very fast.

Experiment: We report the time consumption in Table 1. Experimental results show that our mapping remains almost identical to the original method without significant time loss.

Table 1 time consumption on photo dataset in one epoch

Dear Reviewer bYtP,

We sincerely thank the reviewer for the detailed comments and insightful questions. Our response to your comments one by one as follows.

W1: There is a lack of a more detailed introduction to the Riemannian kernel method, such as how much its distortion is and the influence of the number of basis functions on the mapping. Q1. Could the authors provide more introductions and analyses about this kernel method?

Reply to W1&Q1: There are many studies that use kernel methods or Laplace operators to analyze data manifold learning [1][2][3][4]. This is an interesting mapping method that it essentially preserves the isometric invariance between two points while the feature still remain in Euclidean space. [1] showed that the Laplace eigenfunctions can approximate the estimation of the kernel and replace the exponential mapping in hyperbolic space. Although it is not necessarily mathematically elegant enough, it is a very important property for model training, which means that we can simplify many additional Riemannian operators.

[1] Random Laplacian Features for Learning with hyperbolic space. ICLR 2023

[2] Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning. AAAI 2024

[3] Analysis on Manifolds via the Laplacian.

[4] Analysis of the Laplacian on the complete Riemannian manifold.

---

W2: The author of the manifold signature only considered setting the learnable curvature to obtain it, but did not make a more detailed design for the selection of dimensions and manifolds. Q2: How are the dimensions and manifolds selected?

Reply to W2: As we explained in Section 3.2, for the learnable curvature, we adopt the gyrovector ball as the underlying manifold. Regarding the dimensionality, we currently lack an effective way to optimize it during training and can only set it as a fixed hyperparameter in advance. It is such a valuable comment worth exploring in the future.

---

W3: In order for readers to better understand the background of geometric deep learning[1], more preliminary about Riemannian geometric methods should be added in the article and citations of relevant literature should be supplemented like [2-4] .

Reply to W3: We sincerely appreciate the reviewer’s valuable suggestions. These recommendations are instrumental in strengthening the background of our work. We will carefully incorporate this feedback by expanding the background discussion and adding relevant citations in the revised version.