Riemannian Multiclass Logistics Regression for SPD Neural Networks

Clarification on Deformed Metrics: What was the primary motivation behind introducing the deformed metrics like (θ, α, β)-LEM and (θ)-LCM? Are there specific limitations or challenges with the standard LEM and LCM that these deformations address?

Empirical Validation Scope: The results on datasets like Radar and HDM05 are presented. Could the authors provide insights into why these specific datasets were chosen? Are there plans to test the proposed methods on a broader set of datasets?

Comparison with State-of-the-Art: How does the proposed method compare with the current state-of-the-art methods in SPD manifold learning? Are there benchmark results available that can provide a clearer picture of the paper's standing in the current research landscape?

Hyperparameter Sensitivity: How were the hyperparameters like θ, α, β chosen for the experiments? Is there any empirical analysis available on their sensitivity and impact on the results? This would be crucial for understanding the robustness of the proposed methods.

Intuitive Explanations: Given the depth of mathematical constructs and proofs, could the authors provide more intuitive explanations or real-world examples to make the content more accessible to a broader audience?

Novelty Delineation: It would be beneficial if the authors could explicitly highlight the novel contributions of this work in comparison to existing literature. What are the specific gaps in current research that this paper addresses?

Broader Applications: Are there potential applications of the proposed methods beyond SPD manifold learning in recognition tasks? If so, could the authors discuss or explore these in the paper to enhance its appeal?

Potential for Extensions: Given the introduction of Pullback Euclidean Metrics (PEMs), are there plans or ideas for extending this concept further in future research?

Reproducibility: For the benefit of the research community, could the authors provide details on the availability of code and datasets? This would aid in reproducibility and further exploration by other researchers.

Limitations and Future Work: Every research has its limitations. Could the authors shed light on the potential limitations of their proposed methods and any plans for addressing them in future work?

• In the experimental analyses, the proposed methods perform on par with the baseline and in some cases, it outperforms the baseline. Therefore, they are pretty sensitive to hyperparameter selection. How should the hyperparameters be selected? More precisely, hyperparameter selection procedure was discussed on page 8. In addition to the setup, could you please also explain which algorithms should be used to search for these parameters, and how do selected hyperparameters generalize?

• For different datasets, different hyperparameters provide the best accuracy. What can the reason be?

• How does the accuracy change for different optimizers? Can you provide learning curves for different optimizers?

• Can you theoretically/experimentally compare footprints of the proposed methods and the baseline/sota?

I believe that the main contribution of the paper is the construction of MLR on SPD manifolds associated with variants of Log-Euclidean and Log-Cholesky metrics. However, as I said above, the main paper's weakness is lack of novelty in developing SPD MLR. Its key ingredient here is the reformulation of MLR from the perspective of distance to hyperplanes. This formulation has been used by Ganea et al. (2018) in the context of hyperbolic neural networks, and by Nguyen & Yang (2023) in the context of SPD neural networks. The difference between the proposed reformulation and the one in Nguyen & Yang lies in the definition of the distance from an SPD matrix to an SPD hyperplane in Equation (14). However, this definition is a direct consequence of Theorems 2.23 and 2.24 in Nguyen & Yang (2023), where it is stated that the pseudo-gyrodistance is equal to the SPD gyrodistance (which turns out to be the geodesic distance) in the case of Log-Euclidean and Log-Cholesky metrics. Thus there is no finding in Equation (14).

It is noted in Xavier Pennec et al. (2006) that the Log-Euclidean framework turns SPD manifolds into flat spaces. Thank to this property, extensions of classical methods in Euclidean space to SPD manifolds pose no challenges. Sereval examples are given in Xavier Pennec et al. (2006). Please see also Arsigny et al. (2005) (reference below). This is also the case of Log-Cholesky framework, where it has been shown (Lin, 2019) that the sectional curvature of SPD manifolds under this framework is constantly zero (thus SPD manifolds become flat spaces).

In summary, Equation (14) is a direct consequence of Theorems 2.23 and 2.24 in Nguyen & Yang (2023), and the Log-Euclidean and Log-Cholesky frameworks that the paper focusses on turn SPD manifolds into flat spaces and therefore allow one to straightforwardly derive extensions of MLR to SPD manifolds (Lemma 3.3 and Corollary 4.3).

In terms of experimental results, although the proposed method achieve improvements when it is used in the frameworks of some SPD neural networks, it is hard to see the real benefit of the proposed method in comparison with state-of-the-art methods. For instance, the method in Li et al. (2018) (reference below) achieves 84.47 $\pm$ 1.52 on HDM05 compared to 63.06 $\pm$ 0.76 obtained by the proposed method. The same issue applies to experimental evaluations for radar and EEG classification where more efforts are needed to demonstrate the efficacy of the proposed method compared to state-of-the-art methods. Also, there is no comparison against SPDNetBN which has been shown to outperform SPDNet in a variety of problems.

Questions:

1. Can the proposed method be used in SPDNetBN ? If yes, how about the performance of the resulting network ?

References

1. Arsigny, V., Fillard, P., Pennec, X., and Ayache, N. Fast and Simple Computations on Tensors with Log-Euclidean Metrics. Technical Report RR-5584, INRIA, 2005
2. Li, C., Cui, Z., Zheng, W., Xu, C., Ji, R., and Yang, J.: Action-Attending Graphic Neural Network. IEEE Trans. Image Process. 27(7): 3657-3670 (2018)

In the context of the comments in "Weaknesses" column, please answer the following questions:

1. Please describe the motivation of this work.
2. Please describe its advantages over the gyro-structure based framework.
3. What are the challenges or difficulties of developing the proposed generalisation?
4. What leads to the poor performance of ($\theta$)-LCM in Table 2 (b)?
5. How are the proper values of $\theta$, $\alpha$ and $\beta$ selected in the experimental study?