Algebraic SPD and Correlation Geometry: A Gyro Approach

$\textcolor{red}{\rm{Rebuttal}}$

We thank the reviewer $\textcolor{green}{8a4t(R5)}$ for the instant reply. Below is our detailed response.

1.The theoretical innovations.

Previous works [a,b] have proposed three gyro-structures on SPD manifolds based on Affine-Invariant (AI), Log-Euclidean (LE) and Log-Cholesky (LC) metrics. The Power-Euclidean (PE) metric has demonstrated effectiveness in various applications. Considering its computational efficiency and its convergence to the LE metric as the matrix power approaches zero, we propose a novel SPD gyro-structure under the PE metric. Furthermore, we are the first to introduce two gyro-structures on correlation matrix manifolds. The effectiveness of our approach is validated through extensive experiments on Knowledge Graph Completion (KGC) tasks.

Furthermore, after performing the Cholesky decomposition of the correlation matrix, we observed that each row of the matrix belongs to an open hemisphere and exists an isometry between the open hemisphere and the Poincaré sphere. Building on this, we proposed a novel correlation matrix metric —the Poly Poincaré Cholesky metric (PPCM), and constructed a gyrovector space based on it. Due to time constraints, we conducted experiments on the WN18RR dataset, achieving remarkable results.

Table I: Results on the WN18RR dataset.

3.Symbols and formulae.

In this paper, we propose several gyro-structures based on SPD and correlation matrices under different metrics. Due to the extensive group operations involved, a significant number of symbols and formulas are inevitably introduced. To address this, we will provide a more detailed explanation of the concept of gyrovector spaces and include a comprehensive review of the relevant literature in the introduction. Furthermore, given the dense content of the article, we will optimize the layout to present the material more clearly and make it easier to understand in the final version.

3.The reproducibility of code.

Our experimental code is modified based on [a], whose code is open source. The experimental results can be directly reproduced using the parameter descriptions and specific implementation techniques detailed in the experimental section of the main paper. Additionally, we plan to release our source code in the future.

References

[a] Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds.

[b] Xuan Son Nguyen. A Gyrovector Space Approach for Symmetric Positive Semi-definite Matrix Learning.

$\textcolor{red}{\rm{Rebuttal}}$

We thank Reviewer $\textcolor{red}{L3VE (R4)}$ for the constructive suggestions and insightful comments! In the following, we respond to the concerns in detail.

1.The presentation of the paper.

Thank you for your constructive suggestions. We will provide a more detailed overview of prior works in the introduction section, while clearly explaining the motivation behind our methods and highlighting its distinctions from existing approaches in the final version.

2. The current issues with the method.

In this paper, we propose a novel SPD gyro-structure under the Power-Euclidean (PE) metric and two gyro-structures on correlation matrix manifolds. Our experiments primarily focus on Knowledge Graph Completion (KGC) tasks. Using the proposed gyrovector space, we can also expand other Euclidean networks to manifolds, such as recurrent neural networks (RNN), multinomial logistic regression (MLR), fully connected (FC) and convolutional layers, etc. This is also the limitations of this paper.

3. The choice of manifold and gyro-structure types.

If the data exhibits inherent SPD properties (e.g., covariance matrices), the SPD manifolds should be utilized. Conversely, if the data captures statistical relationships (e.g., correlation matrices), the correlation matrix manifolds are more appropriate. The effectiveness of different gyro-structures may vary depending on the task or dataset. We hope that the proposed SPD gyro-structure under the PE geometry will be a viable alternative to existing Affine-Invariant (AI)-based or Log-Euclidean (LE)-based SPD gyro-structures when these gyro-structures demonstrate unsatisfying performance.

4. The significance of the full-rank correlation matrix.

In Section 4 of the main paper, we discuss the advantages of correlation matrices, which effectively capture more compact statistical information. However, since the metrics defined for SPD matrices cannot be directly applied to correlation matrices, we conducted an in-depth exploration and proposed two novel gyro-structures for full-rank correlation matrices under Euclidean-Cholesky (EC) and Log-Euclidean-Cholesky (LEC) metrics. Experimental results demonstrate that our method outperforms previous methods, further validating its effectiveness. Although our method performs slightly lower than the results under the PE metric, this highlights the suitability of different metrics for different tasks. Moreover, as shown in Table 6 in the main paper, we combine full-rank correlation matrices with SPD and Grassmannian manifolds, achieving superior performance.

$\textcolor{red}{\rm{Rebuttal}}$

We thank Reviewer $\textcolor{blue}{djcf(R3)}$ for the careful review and the suggestive comments. Below, we address the comments in detail.

1.Advantages of proposed methods.

[a] defined two gyros-tructures for SPD manifolds based on the Affine-Invariant (AI) and Log-Euclidean (LE) metrics. The Power-Euclidean (PE) metric has shown remarkable effectiveness across various fields. Given its computational efficiency and its convergence to the LE metric as the matrix power approaches zero, we further propose a gyrostructure based on the power Euclidean metric. Furthermore, we introduce, for the first time, two gyrostructures based on full-rank correlation matrices under the Euclidean-Cholesky (EC) and Log-Euclidean-Cholesky (LEC) metrics. Our experiments demonstrate that this approach outperforms existing AI and LE methods, validating the effectiveness of our proposed framework.

Using gyrovector space, we extend fundamental operations from Euclidean space (such as matrix-matrix addition and scalar-matrix multiplication) to the manifolds, effectively generalizing Euclidean networks to manifolds. Since the core of the KGC task lies in constructing scoring functions, which inherently involve binary operations and matrix scaling (the key operations of gyrovector space), our approach is particularly well-suited for this task. Experimental results demonstrate that our proposed method outperforms existing approaches, further validating its effectiveness.

2.Symbols and formulae.

In this paper, we propose some gyro-structures for SPD and correlation matrices based on different metrics. Given the numerous group operations involved, the more symbols and formulas is unavoidable. To address this, we will provide more detailed explanations and strive to present the content in a clear and comprehensible manner in the final version.

3.Details of experiment.

In the KGC task, [b] first achieved significant results by learning entity embeddings in the Poincaré ball, demonstrating clear advantages over traditional Euclidean space. Then, [c] showed that SPD spaces provide superior geometric information representation. By embedding into SPD manifolds using Affine-Invariant (AI) metrics, further performance improvements were achieved. Subsequently, [a] proposed learning with the LE metric on SPD manifolds, combined with Grassmannian manifolds, leading to enhanced task performance. Therefore, we primarily compare our method with those proposed by [a] and [c], rather than methods based on Euclidean space. Experimental results further validate the effectiveness of our approach.

References

[a] Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds.

[b] Balazevic Ivana，Allen Carl，Hospedales, et al. Multi-relational poincaré graph embeddings.

[c] López Federico，Pozzetti Beatrice, Trettel Steve, et al. Vector-valued distance and gyrocalculus on the space of symmetric positive definite matrices.

$\textcolor{red}{\rm{Rebuttal}}$

We appreciate the constructive comments from Reviewer $\textcolor{purple}{KfSo(R2)}$. The following is our detailed response.

1.Symbol and presentation.

Thank you for your very constructive suggestions! To clarify, we will make corrections based on the identified issues in the final version. We will expand the introduction section to introduce more motivations for our method and highlight its distinctions from previous approaches. Furthermore, we will merge Section 4 with Section 5 and provide more comprehensive details on the KGC task.

2.Difference from prior work.

Previous works [a,b] have proposed three gyro-structures on SPD manifolds based on Affine-Invariant (AI), Log-Euclidean (LE) and Log-Cholesky (LC) metrics. The Power-Euclidean (PE) metric has demonstrated effectiveness in various applications. Considering its computational efficiency and its convergence to the LE metric as the matrix power approaches zero, we propose a novel SPD gyro-structure under the PE metric. Furthermore, we are the first to introduce two gyro-structures on correlation matrix manifolds. The effectiveness of our approach is validated through extensive experiments on Knowledge Graph Completion (KGC) tasks.

3.Experimental performance on full-rank correlation matrices.

we propose two novel gyro structures for full-rank correlation matrices,induced by the theoretically and computationally efficient Euclidean-Cholesky (EC) and Log-Euclidean-Cholesky (LEC) metrics. Although our method performs slightly below the results under the proposed PE metric in Tables 2 and 3 of the main paper, it still outperforms previous methods, demonstrating its effectiveness while highlighting that different metrics are suited to different tasks. Furthermore, in Table 6, we combine full-rank correlation matrices with SPD and Grassmannian manifolds, achieving superior results.

4.Application to other tasks.

Thank you for your valuable suggestions. In the KGC task, distance calculation plays a pivotal role, particularly in constructing the scoring function. Our experimental results demonstrate the effectiveness of the proposed framework. We greatly appreciate your suggestion to explore tasks such as clustering, anomaly detection, and change point detection, which rely heavily on distance metrics. These are insightful directions, and we will consider them in future work to further validate the versatility and robustness of our proposed indicators.

References

[a] Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds.

[b] Xuan Son Nguyen. A Gyrovector Space Approach for Symmetric Positive Semi-definite Matrix Learning.

$\textcolor{red}{\rm{Rebuttal}}$

We thank the reviewer $\textcolor{orange}{uQXC(R1)}$ for the valuable comments. Our detailed responses are given below.

1. Details on the algorithm.

The core of the Knowledge Graph Completion (KGC) task lies in the definition of the scoring function, which we have presented in Section 6.1 of the main paper, specifically in Equation 22. The binary operations and matrix scaling are key operations in the proposed gyrovector space. Details regarding parameter selection, initialization, and specific implementation techniques are further elaborated in Section 6.2. Additionally, we plan to release our source code soon.

2. Correspondence between experimental validation and theoretical support.

In the task of KGC, [a] was the first to achieve significant results by learning entity embeddings in the Poincaré ball, demonstrating clear advantages over traditional Euclidean spaces. Building on this, [b] found Symmetric Positive Definite (SPD) spaces offer superior geometric information representation. By embedding into SPD manifolds based on Affine-Invariant (AI) metrics, further improvements were achieved. Subsequently, [c] proposed learning with the Log-Euclidean (LE) metric on SPD manifolds, combined with Grassmannian manifolds, achieving enhanced performance.

Building on these advancements, we propose a novel approach to learning on SPD manifolds under the Power-Euclidean (PE) metric and also introduce learning with Euclidean-Cholesky(EC) and Log-Euclidean-Cholesky (LEC) metrics on full-rank correlation matrix manifolds, combined with Grassmannian manifolds to achieve superior results. Since the score function of KGC involves matrix addition and matrix scaling operations, which are the key operations of our proposed gyrovector space, the KGC task has important theoretical relevance to our method.

3. Application of the proposed method in other fields.

In this paper, we propose a novel SPD gyro-structure under the PE metric and two new gyro-structures for full-rank correlation matrices under the Euclidean-Cholesky (EC) metric and the Log-Euclidean-Cholesky (LEC) metrics. In the KGC task, we apply our theoretical framework to construct a scoring function, achieving effective results. Furthermore, the proposed theory has the potential to extend other Euclidean networks to manifolds, enabling applications in tasks such as text or image classification. At the same time, this is also the limitation of this paper and a direction for future research.

References

[a] Balazevic Ivana，Allen Carl，Hospedales, et al. Multi-relational poincaré graph embeddings.

[b] López Federico，Pozzetti Beatrice, Trettel Steve, et al. Vector-valued distance and gyrocalculus on the space of symmetric positive definite matrices.

[c] Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds.