Towards a General Attention Framework on Gyrovector Spaces for Matrix Manifolds

We thank Reviewer $\textcolor{purple}{\rm{3rxG (R4)}}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😊

1. GyroAtt yields consistent and statistically significant improvements over baselines.

Table A: Comparison between GyroAtt and the baseline MAtt, where * denotes $p$-vale < 0.05 and n.s. denotes "not significant".

We emphasize that the observed performance gains are both substantial and consistent across datasets. As shown in Table A, four out of six cases show statistical significance at $p < 0.05$, based on paired t-tests across cross-validation folds. Importantly, GyroAtt consistently outperforms MAtt across all datasets, with a notable $+14.8\%$ gain in the inter-subject setting, which is particularly challenging in EEG decoding. While some confidence intervals do overlap, this is expected due to the intrinsic characteristics of EEG data, including low signal-to-noise ratios (SNR), significant inter-/intra-session domain shifts, and high variability in subject responses—all of which naturally widen confidence intervals. Despite this high-variance regime, the consistent trend and statistical significance in most cases support the robustness of GyroAtt.

2. Advantages of Gyro Hom over manifold MLP‑like layers

Table B: Ablation on transformation layers (mean ± std, %). Here, / means that the method cannot converge.

2.1 Comparison with SPD-FC Layers.

As reported in Table 8, we benchmark BiMap [a] as the canonical MLP‑like transformation on SPD. For a broader comparison, we also include SPD‑FC‑LE/LC [b], which implements manifold fully connected (FC) layers under Log‑Euclidean/Log‑Cholesky metrics. Taking the Log-Euclidean FC layer (SPD-FC-LE) as an example, let $v _{(i,j)}(X) = \langle \ominus _{le} P _{(i,j)} \oplus _{le} X, W _{(i,j)} \rangle ^{le}, P _{(i,j)}, W _{(i,j)} \in \text{Sym} _n ^{+,le},i \leq j,i,j = 1,\ldots,m$. Then the output of an FC layer is computed as $Y = \mathrm{expm}([y _{(i,j)}] _{i,j=1} ^m)$, where $y _{(i,j)}$ is given by

We now show that SPD-FC-LE is essentially a Euclidean FC layer in the log domain and corresponds to a special case of Gyro Hom under LEM. Here, the LE addition reduces to Euclidean operations in the log domain. Specifically,

so

where $b_{(i,j)} = -\langle \mathrm{logm} P_{(i,j)}, \mathrm{logm} W_{(i,j)} \rangle$. This shows that SPD-FC-LE is exactly a Euclidean FC layer applied in the log domain, followed by a matrix exponential:

Importantly, when $b_{(i,j)} = 0$, i.e., $P_{(i,j)} = I$, the transformation becomes a linear map. In this case, $F$ satisfies the gyro homomorphism:

which confirms that SPD-FC-LE is a special case of Gyro Hom under LEM. Similarly, SPD-FC-LC is a special case under the LCM metric.

Only replacing Gyro Hom with BiMap or SPD‑FC‑LE/LC while keeping all other architectural components unchanged, we observe that Gyro‑Hom consistently outperforms SPD‑FC‑LE/LC across four datasets (Table B). In addition, Gyro‑Hom is significantly more efficient in computational complexity. Recall that each $v_{(i,j)}$ in SPD-FC-LE requires a pair of symmetric matrices $(P_{(i,j)}, W_{(i,j)}) \in \mathrm{Sym}(n)$, leading to $n(n+1)$ parameters per component. For $S^d\to S^d$, this amounts to $\tfrac{1}{2} n^2(n+1)^2$ parameters in total. Moreover, the computational complexity is $\mathcal{O}(n^4)$ per forward pass, in addition to the $\mathcal{O}(n^3)$ cost of matrix logarithms and exponentials. In contrast, our Gyro Hom uses only an $n\times n$ parameter with $n^2$ parameters and $\mathcal{O}(n^3)$ flops (matrix multiplications and matrix logarithms and exponentials), matching the observed wall‑clock savings.

2.2 Comparison with BiMap.

To further highlight the benefits of our method beyond efficiency, we analyze its theoretical advantages over BiMap.

• Gyro Algebraic Structure Simplifies Manifold Layer Design: Gyro spaces define operators such as gyro addition $\oplus$ and scalar multiplication $\otimes$, providing an algebraic framework that is both Euclidean-like and geometry-aware. This facilitates the construction of manifold layers while respecting intrinsic geometry.
• Respecting Gyro Algebraic Structure via Gyro Homomorphisms: GyroAtt employs gyro homomorphisms in its transformation layers, preserving the underlying algebraic structure and Riemannian geometry. In contrast, MAtt and GDLNet rely on BiMap/FrMap, which only partially adhere to Riemannian geometry. While these layers enforce manifold constraints, they lack isometric properties, potentially distorting the intrinsic geometric structure. For example, the distance $d(S_1, S_2)$ between two SPD matrices $S_1$ and $S_2$ may not be preserved after transformation, i.e., $d(\mathrm{BiMap}(S_1), \mathrm{BiMap}(S_2)) \neq d(S_1, S_2)$. This limitation undermines their ability to fully respect and retain the original geometric properties of the data, which can negatively impact performance.

3. Insights and theoretical advantages

Table B: Comparison between the GyroAtt and Euclidean Attention.

• Theoretical generality: Gyrovector spaces naturally generalize Euclidean vector spaces. As shown in Table B, each GyroAtt component extends its Euclidean counterpart in a mathematically grounded way.
• Natural Extension: When the manifold is standard Euclidean geometry, G1 and G3 in GyroAtt exactly reduce to E1 and E3 in Euclidean Attention.
• Scalability across geometries: The unified formulation allowed us to instantiate GyroAtt over seven variants (three SPD, three SPSD, one Grassmann) under the consistent network archetecture, highlighting the applicability of the framework across diverse settings.

4. Principled identification of gyro homomorphisms

Thanks for the constructive comments. There are indeed a principled way to identify the gyro homomorphism. As gyrovector space over the manifold is normally defined on the Riemannian homogeneous space [Sec. 3, c], let us focus on the homogeneous space.

Theorem A (Sufficient Condition) Let $\mathcal{M}$ be a Riemannian homogeneous space whose group of isometries is $G$. Besides, $(\mathcal{M},\oplus,\otimes)$ forms a gyrovector space, where $\oplus$ and $\otimes$ are defined as [Eqs. (1-3), d]. Let $e$ be the gyro identity element. Every isometry $f \in G$ that fixes the identity $f(e) = e$ is a gyro homomorphism.

Intuitively, we can identify a homomorphism from the isometries, which are known for a given homogeneous space. This theorem applies to all known matrix gyrovector spaces (e.g., SPD, SPSD, and Grassmannian). For example, one the SPD manifold under the AIM, $GL(n)$ is the isometry group, each of which characterize an isometry, $f _A : \mathcal{S} _{++} ^n \in S \mapsto A S A ^\top \in \mathcal{S} _ {++} ^n$, with $A \in GL(n)$. Applying $f _{A}(I)=I$ implies that $A$ should be an orthogonal matrix. This is exactly the result in Thm. 4.1.

Besides, if a manifold has a flat structure, we can directly identify the gyro homomorphism from the flat structure. This is how we obtain the homomorphism for the SPD manifold under LEM and LCM (Thm. 4.1). Specifically, for the LEM, the SPD manifold becomes a flat space under the matrix logarithm. This allows us to construct a gyro homomorphism via linear operations in the log domain. In this case, the homomorphism reduces to the form

which matches the second case in Eq. (8) of Theorem 4.1.

Proof of Thm. A This theorem can be induced from Lem. 2.1-2.2 in [d]. Let the Riemannian metric tensor be $g$. We have

where $f^*g$ is the pullback metric under the diffeomorphism $f$. As $f$ is an isometry, we have $f^*g=g$. Applying Lem. 2.1-2.2 in [d], one can immediately prove $f(P \oplus Q) = f(P) \oplus f(Q)$ and $f(t \otimes Q) = t \otimes f(Q)$ for any $t \in \mathbb{R}$ and $P,Q \in M$.

References

[a] A Riemannian network for SPD matrix learning.

[b] Matrix Manifold Neural Networks++.

[c] The Gyro-Structure of Some Matrix Manifolds.

[d] Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach.

Dear Reviewer 3rxG,

We sincerely thank you for the thoughtful comments and the time dedicated. Due to the number of follow-up experiments required to address the two main points raised, our response was submitted slightly later. Based on your feedback, we have carefully addressed the key concerns as follows:

1. Gyro-Homomorphism over the Stereographic Model: We show that our gyro-homomorphism naturally extends to the stereographic model and is curvature-independent.

2. Additional Comparisons with MLP-like Layers: We added experiments comparing GyroHom with more SPD and Grassmannian MLP-like transformations.

We hope these additions (please see our responses in Parts 1, 2, and 3) further clarify the theoretical scope and empirical effectiveness of our approach. If you have any further questions or suggestions, we would be more than happy to continue the discussion.

Thanks again

Warm regards

We substitute the gyro-homomorphism (GyroHom) layer in SPD GyroAtt with the existing SPD MLP-like layers, such as BiMap, SPD FC, and Tangent-FC. We compare them under LEM, LCM, and AIM metrics.

• BiMap. Across all datasets and metrics, GyroHom outperforms BiMap in accuracy. As shown in Table E, under the AIM metric, GyroHom achieves +7.8% improvement in the BNCI2014001 inter-subject task (53.1% vs. 45.3%) and +3.8% in the BNCI2015001 inter-subject setting (77.9% vs. 74.1%).

• Tangent-FC. As shown in Tables C–E, Tangent-FC fails to converge in several settings and underperforms GyroHom by large margins where results are available (e.g., 42.4% vs. 75.4% under AIM, inter-session). Since both LEM and AIM employ the same Log and Exp maps at the identity, Tangent-FC yields identical results under these two metrics. Therefore, we report it only once. GyroHom is significantly more stable and effective than Tangent-FC.

• SPD-FC. While SPD-FC can achieve competitive accuracy, its computational cost is prohibitive. For each mapping $S^d \to S^d$, the SPD-FC-AIM layer requires performing $2d(d+1)$ SVDs of size $d \times d$ per forward pass, resulting in N/A due to excessive runtime. In contrast, GyroHom achieves better accuracy with significantly lower runtime e.g., 4.44s vs. 223.27s under AIM, BNCI2015001 inter-session.

• MAtt. As shown in Table D, GyroHom also outperforms the widely used MAtt baseline across all tasks. Under the inter-subject settings, GyroHom improves performance by a notable margin:+7.8% on BNCI2014001 (53.1% vs. 45.3%), +14.8% on BNCI2015001 (77.9% vs. 63.1%).

These results confirm that GyroHom offers a superior trade-off between accuracy, efficiency, and stability compared to existing SPD manifold layers.

2.2 Grassmannian Manifold Comparison

Table E: Ablation on transformation layers (mean ± std, %). Here, / means that the method cannot converge.

We additionally evaluate GyroAtt on the Grassmannian manifold by comparing it with two MLP-like transformation layers: FrMap+ReOrth [b, Secs. 3.1-2], Tangent-FC and Gr-Scaling [c, Sec. 3.2]. Gr-Scaling is defined as:

where:$A \in \mathbb{R} ^{p \times (n-p)}$, $B \in \mathbb{R} ^{(n-p) \times p}$. $P$ is defined as:

As shown in Table E, GyroAtt consistently achieves higher accuracy, lower runtime, and better stability across all tasks.

• GDLNet. GyroAtt achieves much higher accuracy than GDLNet across all tasks ,e.g., +14.4% on BNCI2014001 inter-session, +12.0% on BNCI2015001 inter-subject, with comparable runtime.
• FrMap+ReOrth. GyroAtt consistently outperforms FrMap across all tasks, with +5.3%, +3.1%, and +3.0% improvements on BNCI2014001 inter-session, BNCI2015001 inter-session, and BNCI2015001 inter-subject, respectively.
• Tangent-FC. GyroAtt is both faster and more accurate. On BNCI2014001 inter-subject, it achieves +5.8% higher accuracy (52.1% vs. 46.3%) with nearly 2× lower runtime (100.2s vs. 188.3s), and shows slightly lower variance.
• Gr-Scaling. Gr-Scaling fails to converge on inter-subject tasks and performs worse where it does run (79.2% vs. 85.0%). It also incurs higher cost (9.68s vs. 5.28s), indicating lower efficiency.

GyroAtt achieves the best overall trade-off between accuracy, speed, and stability, outperforming all baselines on the Grassmann manifold.

References

[a] Constant curvature graph convolutional networks

[b] Building deep networks on Grassmann manifolds

[c] Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach

[d] Matrix Manifold Neural Networks++

[e] A Riemannian network for SPD matrix learning

[f] MAtt: A manifold attention network for EEG decoding

2. More Comparisons with MLP-like Layers

Table A: Summary of differrnt MLP-like layers on the SPD Manifold

Table B : Summary of differrnt MLP-like layers on the Grassmannian Manifold

We appreciate the reviewer’s insightful suggestions. Since the SPSD manifold currently lacks established MLP-like layers, we compare our gyro homomorphism against different MLP-like layers on the SPD and Grassmann manifolds. Following the Möbius-type matrix-vector multiplication [a, Defs. 1-2], we also construct the Tangent-FC map $F:\mathcal{M} \to \mathcal{M}$ via the tangent space at the identity:

where $\mathrm{Log} _{I}(\cdot)$ and $\mathrm{Exp} _{I}(\cdot)$ are the Riemannian Log and Exp taken at the identity, $\mathrm{vec}(\cdot)$ the vectorisation, and $\operatorname{mat}(\cdot)$ its inverse that reshapes a vector back to a matrix.

We summarize the experimental results below. Please refer to the following sections for detailed comparisons.

• SPD Manifolds: GyroAtt-SPD equipped with the GyroHom layer consistently delivers the highest accuracy across all SPD-based transformation layers, while significantly reducing runtime compared to SPD-FC and SPD-Tangent-FC.
• Grassmannian Manifolds：GyroAtt-Gr with the GyroHom layer achieves superior accuracy over all Grassmannian transformations, and demonstrates better efficiency than Gr-Scaling and Gr-Tangent-FC.

2.1 SPD Manifold Comparison

Table C: Ablation on transformation layers (mean ± std, %) for SPD manifolds under LEM. Here, / means that the method cannot converge.

Table D: Ablation on transformation layers (mean ± std, %), for SPD manifolds under LCM. Here, / means that the method cannot converge.

Table E: Ablation on transformation layers (mean ± std, %), for SPD manifolds under AIM. Here, / means that the method cannot converge. “N/A”: not completed due to SPD-FC’s $\mathcal{O}(d^5)$ cost per forward pass.

Table F: Comparison of MAtt (mean ± std, %).

We sincerely thank you for the thoughtful comments, constructive suggestions, and the time dedicated to reviewing our paper.

1. Homomorphism over the stereographic model

We sincerely thank the reviewer for their insightful comments. The stereographic model $\mathfrak{st}_\kappa^d$ also admits gyrovector structure [a, Eqs.(2-3)]. Therefore, we can readily identify its gyro-homomorphism by our Theorem A. It turns out that the homomorphism is independent of curvature. This is not surprising, as gyroaddition and scalar gyromultiplication share the same expressions, except for the curvature parameter $\kappa$.

In particular, Theorem 12 in [a] shows that every isometry $\phi: \mathfrak{st} _\kappa ^d \to \mathfrak{st} _\kappa ^d$ over the stereographic model can be written as

Wherein, $\mathrm{O}(d)$ is the group of $d \times d$ orthogonal matrices, and $\oplus _\kappa$ represents the gyroaddition. By applying Theorem A,

we derive the following gyro-homomorphism for $\mathfrak{st} _\kappa ^d$:

Importantly, this expression is independent of the curvature $\kappa$, applicable to Euclidean ($\kappa = 0$), hyperbolic ($\kappa < 0$), and spherical ($\kappa > 0$) settings.

Please note that in GyroAtt, the curvature parameter $\kappa$ still plays a critical role in other components of the attention mechanism. Since gyro distance is curvature-dependent, $\kappa$ directly affects both the query–key similarity computation and the weighted Fréchet mean (WFM) aggregation.

We thank Reviewer $\textcolor{red}{2oRe (R2)}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😄

1. GyroAtt is applicable to a wide range of manifold learning tasks beyond EEG.

Thank you for the insightful question. While our experiments focus on EEG decoding, the GyroAtt framework is broadly applicable to learning tasks involving manifold-valued data. Data that lie on matrix manifolds—such as SPD, SPSD, or Grassmannian manifolds—are ubiquitous in various application domains, such as medical imaging [a], shape analysis [b], drone classification [c], image recognition [d], human behavior analysis [e, f], EEG [g], graph classification [h], and knowledge graph completion (KGC) [i, j].

To further support its generalizability, we include two additional experiments beyond EEG:

Graph Node Classification. We evaluate GyroAtt-SPD on two benchmark datasets—Pubmed and Cora—using SPD-GAT [h] as the base model, where the standard attention module is replaced with either GyroAtt-SPD or MAtt. Results are summarized below:

Drone Recognition. We further test GyroAtt on the Radar dataset for drone classification, following the GDLNet [g] architecture. The results are as follows:

GyroAtt-SPD again achieves the highest accuracy, further validating its adaptability to non-EEG modalities such as radar signals.

2. We will release the full GyroAtt implementation

Apologies for the ambiguity. We fully intend to release the complete GyroAtt implementation, including training pipelines and model definitions, alongside the data access code. The sentence in Line 1051 will be updated in the final version to reflect this more clearly.

References

[a] Fast and Simple Computations on Tensors with Log-Euclidean Metrics.

[b] Covariance Descriptors for 3D Shape Matching and Retrieval.

[c] Riemannian Batch Normalization for SPD Neural Networks.

[d] DeepKSPD: Learning Kernel-Matrix-Based SPD Representation for Fine-Grained Image Recognition.

[e] Deep Manifold Learning of Symmetric Positive Definite Matrices with Application to Face Recognition.

[f] GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition.

[g] MAtt: A Manifold Attention Network for EEG Decoding.

[h] Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices.

[i] The Gyro-Structure of Some Matrix Manifolds.

[j] Matrix Manifold Neural Networks++.

Thank you for your prompt and positive reply! If you have any further suggestions or insights, we would be very glad to hear them! 😄

We thank Reviewer $\textcolor{orange}{UbxM (R3)}$ for the constructive suggestions and insightful comments! In the following, we respond to the concerns in detail. 😄

1. The limitations of previous work

In Euclidean spaces, basic layers like convolution and fully connected operations provide a flexible basis for diverse network designs. However, in the context of manifold-valued data, there are no counterparts to these foundational layers due to geometry-specific constraints. This limits the flexibility of layer design and hinders reuse across different manifolds. Although deep learning models are often treated as black-box systems, it is essential in such settings to consider whether these layers (i) preserve the input’s geometric structure and (ii) effectively leverage the underlying Riemannian geometry.

Preserving geometric structure. The commonly used BiMap layer is inherently restricted to SPD matrices of shape $d \times d$ and operates as:

where $X \in S^d$ and $W \in \mathbb{R}^{d \times k}$. A fundamental requirement of this operation is that the input must be square. However, for data lying on the Grassmannian manifold, such as subspace features $Y \in \mathbb{R}^{d \times q}$, this assumption no longer holds. Since $Y$ is rectangular and not square, BiMap becomes ill-defined and cannot be directly applied.

Conversely, the FrMap layer, used in Grassmannian settings, applies a transformation of the form:

which is compatible with Grassmannian data. Although SPD matrices $X \in S^d$ can technically be input to this operation, it destroys the key structural properties: $WX$ is generally not symmetric and not positive definite, thus violating the manifold constraint and pushing the data off the SPD manifold. In summary, these geometry-specific designs lack portability and generality and are infeasible outside their intended manifolds.

Additionally, geometry-specific layers lead to fragmented architectures with low reusability. This increases implementation overhead. In contrast, GyroAtt adopts a unified architecture that seamlessly supports SPD, SPSD, and Grassmann manifolds without redesign, enabling consistent and modular learning across diverse geometries.

Leveraging Riemannian geometry. From a theoretical standpoint, Gyro Hom offers three key advantages:

• Gyro Algebraic Structure Simplifies Manifold Layer Design: Gyro spaces define operators such as gyro addition ⊕ and scalar multiplication ⊗, providing an algebraic framework that is both Euclidean-like and geometry-aware. This facilitates the construction of manifold layers while respecting intrinsic geometry.
• Respecting Gyro Algebraic Structure via Gyro Homomorphisms: GyroAtt employs gyro homomorphisms in its transformation layers, preserving the underlying algebraic structure and Riemannian geometry. In contrast, MAtt and GDLNet rely on BiMap/FrMap, which only partially adhere to Riemannian geometry. While these layers enforce manifold constraints, they lack isometric properties, potentially distorting the intrinsic geometric structure. For example, the distance $d(S_1, S_2)$ between two SPD matrices $S_1$ and $S_2$ may not be preserved after transformation, i.e., $d(\mathrm{BiMap}(S_1), \mathrm{BiMap}(S_2)) \neq d(S_1, S_2)$. This limitation undermines their ability to fully respect and retain the original geometric properties of the data, which can negatively impact performance.
• Nonlinear Activation: Existing SPD networks often use ReEig for nonlinear activation, which enforces SPD-ness through eigenvalue regularization, but lacks geometric interpretability. In contrast, GyroAtt adopts a matrix power-based nonlinear activation function that essential activates the latent Riemannian geometry. As shown in [a, Fig. 1], the matrix power can deform any given Riemannian metric. For instance, as the parameter $\theta \to 0$, the matrix power deform AIM into LEM.

2. Clarification on Experimental Setup and Evaluation Metrics

We thank the reviewer for this important question. Our experimental setup closely follows the protocols established in prior works, and we clarify the details as follows:

• For MAMEM-SSVEP-II and BCI-ERN datasets, we strictly adhere to the evaluation setup and metrics described in MAtt [b]. Specifically, we use sessions 1–3 for training, session 4 for validation, and session 5 for testing. We also adopt the same evaluation metrics as in [b]: accuracy for MAMEM-SSVEP-II and AUC for BCI-ERN to account for class imbalance.

• For BNCI2014001 and BNCI2015001 datasets, our evaluation is based on the protocol in [c]. We adopt their inter-session and inter-subject evaluation splits and use the same performance metric: balanced accuracy, computed as the average recall across all classes.

• Regarding preprocessing, we follow [b] for MAMEM-SSVEP-II and BCI-ERN, and [c] for BNCI2014001 and BNCI2015001 to ensure consistency and reproducibility. Details of the used datasets and preprocessing steps are provided in App.D.1.

We will revise the manuscript to explicitly clarify these details and remove any ambiguity around our evaluation protocol.

References

[a] RMLR: Extending Multinomial Logistic Regression into General Geometries.

[b] MAtt: A manifold attention network for EEG decoding.

[c] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG.

Dear Reviewer UbxM,

We sincerely thank you again for your valuable comments. We hope our previous response has addressed your concerns regarding the limitations of prior works and the design of our experimental setup and evaluation metrics.

As the discussion phase is drawing to a close, we would like to kindly check whether our response has addressed the points you raised. If there are any remaining issues or suggestions, we would be more than happy to provide further clarification. Your feedback is truly appreciated and would be instrumental in helping us further improve the quality of the paper.

Warm regards

We thank Reviewer $\textcolor{blue}{\rm{5ZCS (R1)}}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😊

1. Statistical Significance of Reported Improvements.

Table A: Comparison between GyroAtt and the baseline MAtt, where * donetes $p$-vale < 0.05 and n.s. denotes "not significant".

While some confidence intervals overlap, the improvements are substantial, consistent across all benchmarks. Particularly in the inter-subject setting, which is inherently more challenging, a +14.8% gain highlights the practical effectiveness of GyroAtt. We will clarify the statistical testing procedure in the final version.

2. Experimental comparison of time efficiency.

Table B: Training time (s/epoch) comparison of different manifold attention networks on the BNCI2014001 and BNCI2015001 datasets.

For practical comparison, we measured the average training time per epoch (seconds) for our GyroAtt variants and baseline models MAtt and GDLNet on the BNCI2014001 and BNCI2015001 datasets. As shown in Table B, GyroAtt-Gr is slower than GDLNet, mainly due to additional SVD operations required by the WFM aggregator. Similarly, GyroAtt-SPSD is less efficient than both baselines across all metrics. In contrast, GyroAtt-SPD with the LCM metric achieves slightly better runtime than MAtt. In general, LCM-based GyroAtt variants are faster than their AIM- and LEM-based counterparts, since Cholesky decomposition (used in LCM) is more efficient than SVD-based operations. AIM is consistently slower than LEM due to the higher computational cost of eigenvalue computations. Although some GyroAtt variants introduce additional computational overhead compared to baseline methods, especially under certain metrics, the performance improvements justify the increased complexity. Selecting efficient metrics like LCM can mitigate computational costs, making GyroAtt practical for real-world applications. Please refer to Appendix D.7 for further implementation and profiling details.

3. Analysis of Parameter Counts.

Table C: Learnable parameters (in millions) of different methods across four EEG datasets.

For a manifold attention layer mapping from $S^d \to S^d$, the number of parameters in GyroAtt-SPD is: $\frac{3}{2}n(n-1) + \frac{1}{2}n(n+1)=2n ^2-\frac{1}{2}n$, where the first term corresponds to the transformation layers for $Q$, $K$, and $V$, and the second term represents the bias parameters. In comparison, the baseline MAtt uses $3n^2$ parameters per attention block.

As summarized in Table C, all GyroAtt variants have comparable or even fewer total learnable parameters than MAtt across four EEG datasets, and are significantly more compact than GDLNet, particularly on MAMEM-SSVEP-II. Importantly, GyroAtt consistently achieves better performance despite this compactness, indicating that the gains are not simply due to increased model capacity.

4. The glossary for abbreviations.

We have included a glossary of notations and abbreviations in Appendix B, specifically in Tables 11 and 12, to improve readability and reduce the need to reference earlier sections when encountering abbreviations.

5. Clarification on Initialization Strategy in Section C.3

We appreciate the reviewer’s suggestion regarding the use of a common principal components algorithm (CPCA) for initialization. We would like to clarify a potential misunderstanding:

In Line 2 of Algorithm 6, we initialize the common subspace $U^m$ with a fixed canonical basis $\widetilde{I}_{n,q}$, not via decompositions of individual samples. This line is intended to indicate that the default initialization of the Grassmannian Fréchet mean starts at the identity point on the manifold. We acknowledge that this notation may have caused confusion and will revise it in the final version for clarity.

Importantly, the estimation of the common subspace is learned dynamically in the subsequent steps. Specifically, we first compute the Grassmannian Fréchet mean of the individual subspaces $\{ U_i \}$ obtained via SVD of each sample, and then update $U^m$ via geodesic interpolation toward this mean, and the same choice is also adopted in [a].

6. Comparison with SOTA Baselines.

Our main baselines are strong SOTA methods specifically designed for manifold-valued EEG data:

• On the MAMEM-SSVEP-II and BCI-ERN datasets, our main baselines are GDLNet [b] and MAtt [c], both of which represent state-of-the-art approaches for SPD and Grassmannian manifolds, respectively.
• On the BNCI2014001 and BNCI2015001 datasets, we include TSMNet [d] and Graph-CSPNet [e], both of which are recent SOTA models widely used in motor imagery tasks. Notably, Graph-CSPNet [e] is a recent GNN-based approach tailored for motor imagery classification.

To ensure a fair comparison, we also follow the exact same data preprocessing and evaluation protocol as used in the original baseline papers: For MAMEM-SSVEP-II and BCI-ERN datasets, our setting matches that of MAtt [c]; For BNCI2014001 and BNCI2015001 datasets, we follow the exact pipeline from TSMNet [d].

7. Comparison with Graph Neural Networks.

7.1 Comparison with Graph-Based EEG Methods.

Table D: Accuracy (%) comparison between Graph-CSPNet and GyroAtt on EEG datasets.

We compare GyroAtt against Graph-CSPNet [e], a recent SPD-based GNN method designed for motor imagery classification. As shown in the table below, all variants of GyroAtt outperform Graph-CSPNet across both inter-session and inter-subject settings on BNCI2014001 and BNCI2015001.

7.2. Additional Comparison on Graph Attention Neural Networks.

Table E: Accuracy (%) comparison of SPD-GAT-GyroAtt under graph node classification.

To further validate GyroAtt, we compare GyroAtt against hyerpbolic and SPD attention in standard GNN settings using graph node classification tasks.

We focus on the graph attention (GAT) setting. H-GAT [f] serves as a hyperbolic graph attention baseline, while SPD-GAT [g] is the SPD counterpart. By replacing SPD-GAT’s attention layer with MAtt and GyroAtt, we demonstrate that our attention module consistently improves performance. This highlights the versatility of GyroAtt across both manifold-valued and graph-structured domains.

8. Computation Time and Convergence on Large Matrices.

Table F: Scalability of GyroAtt-SPD on BNCI2015001 with varying input matrix sizes.

We further conducted scalability experiments using GyroAtt-SPD on the BNCI2015001 dataset by adjusting the output dimensionality of the feature extraction layer, thereby increasing the size of the input SPD matrices from $43 \times 43$ to $100 \times 100$. As shown in Table F, the per-epoch time increases moderately with matrix size, and convergence is consistently achieved within a practical timeframe. These results confirm that GyroAtt remains computationally feasible even for relatively large matrices, in agreement with theoretical expectations.

In terms of accuracy, we observe that increasing the matrix size from 43 to 100 results in only marginal variations in performance. This suggests a saturation effect often seen in low-dimensional EEG tasks, where expanding the representational capacity beyond a certain point offers little additional discriminative power. The relevant signal information is likely already captured at smaller sizes, which accounts for the stable test accuracy across the entire range.

References

[a] Matrix Manifold Neural Networks++.

[b] A Grassmannian Manifold Self-Attention Network for Signal Classification.

[c] MAtt: A manifold attention network for EEG decoding.

[d] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG.

[e] Graph neural networks on SPD manifolds for motor imagery classification: A perspective from the time-frequency analysis.

[f] Hyperbolic graph attention network.

[g] Modeling graphs beyond hyperbolic: graphneural networks in symmetric positive definite matrices.

Dear reviewer, we have revised our manuscript with glossaries-extra package, which can automatically generate hyperlinks for each acronym to their definitions. This will improve clarity and make the manuscript easier to navigate. Although we cannot upload a new PDF during the rebuttal, this will be reflected in the final version.