Deep Geodesic Canonical Correlation Analysis for Covariance-Based Neuroimaging Data

Our response (Part 2):

Rebuttal Table 2: Auxiliary multi-view EEG downstream results summarize the statistical results in Table 2 in the manuscript. We used permutation, paired t-tests to identify significant differences between the lower-bound (i.e., no pre-training) with DeepGeoCCA for three scenarios ($S_1$, $S_2$, $S_1 \rightarrow S_2$; t-max adjustment for multiple comparisons) and the KU (df=53, 1e4 permutations) and BNCI datasets (df=11, exhaustive permutations).

References:

[1] S. H. Fairclough and F. Lotte, “Grand Challenges in Neurotechnology and System Neuroergonomics,” Front. Neuroergonomics, vol. 1, p. 602504, Nov. 2020, doi: 10.3389/fnrgo.2020.602504.

[2] T. Warbrick, “Simultaneous EEG-fMRI: What Have We Learned and What Does the Future Hold?,” Sensors, vol. 22, no. 6, p. 2262, Mar. 2022, doi: 10.3390/s22062262.

[3] M. G. Philiastides, T. Tu, and P. Sajda, “Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI,” Annu. Rev. Neurosci., vol. 44, no. 1, pp. 315–334, Jul. 2021, doi: 10.1146/annurev-neuro-100220-093239.

[4] X. Liu, T. Tu, and P. Sajda, “Inferring latent neural sources via deep transcoding of simultaneously acquired EEG and fMRI.” arXiv, Nov. 2022. http://arxiv.org/abs/2212.02226

We appreciate the reviewers' effort to provide feedback and suggestions. Thank you also for expressing your concerns regarding weaknesses and posing additional questions. Please find our detailed responses below.

Weakness 1: The authors proposed a metric and a learning framework for "covariance-based data" on symmetric positive definite manifolds, but they have not yet motivated why the EEG/fMRI data are “covariance-based data” and/or why they lie on symmetric positive definite manifolds. While this might be obvious to some people, it is a bit confusing to me.

Our response: Thank you for pointing us to this issue. Based on the feedback of this reviewer, we included a precise motivation for covariance-based data in Appendix A. Specifically, we added motivations for EEG spatial covariance matrices and fMRI functional connectivity matrices as elements on the SPD manifolds.

Firstly, let us formally define covariance-based neuroimaging data as follows:

Definition (Covariance-Based Neuroimaging Data): the covariance matrix of a neuroimaging data segment, represented by $X \in \mathbb{R}^{n_C \times n_T}$, is denoted as $S = XX^{\top} \in \mathcal{S}_{++}^{n_C}$, where $n_C$ is the number of spatial dimensions and $n_T$ is the signal timesteps.

By definition, covariance-based data is inherently symmetric and positive definite, existing on the Symmetric Positive Definite (SPD) manifolds. Even for degenerate cases (e.g., co-linear spatial dimension), the positive definiteness can be ensured via shrinkage regularization or dimensionality reduction.

Next, let us briefly explain why we chose the second-order statistics of these neuroimaging data as our primary focus for investigation. In a nutshell, EEG spatial covariance matrices capture relevant aspects of neural activity within short to long-time scales. Riemannian geometry-aware methods operating with these covariance-based data are considered state-of-the-art in various EEG application domains, including clinical applications [1,2] and recent brain-computer interface competitions [3,4]. Concerning fMRI, covariance (or correlation) matrices between regions of interest reflect static and dynamic communication between brain networks and their alterations in tasks [5] and pathological conditions [6].

References:

[1] D. Sabbagh, P. Ablin, G. Varoquaux, A. Gramfort, and D. A. Engemann, “Predictive regression modeling with MEG/EEG: from source power to signals and cognitive states,” NeuroImage, vol. 222, p. 116893, Nov. 2020, doi: 10.1016/j.neuroimage.2020.116893.

[2] L. A. W. Gemein et al., “Machine-learning-based diagnostics of EEG pathology,” NeuroImage, vol. 220, p. 117021, Oct. 2020, doi: 10.1016/j.neuroimage.2020.117021.

[3] R. N. Roy et al., “Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation,” Front. Neuroergonomics, vol. 3, p. 838342, Apr. 2022, doi: 10.3389/fnrgo.2022.838342.

[4] X. Wei et al., “2021 BEETL Competition: Advancing Transfer Learning for Subject Independence and Heterogenous EEG Data Sets,” in Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, D. Kiela, M. Ciccone, and B. Caputo, Eds., in Proceedings of Machine Learning Research, vol. 176. PMLR, Dec. 2022, pp. 205–219.

[5] H.-J. Park and K. Friston, “Structural and Functional Brain Networks: From Connections to Cognition,” Science, vol. 342, no. 6158, p. 1238411, Nov. 2013, doi: 10.1126/science.1238411.

[6] A. Yamashita et al., “Generalizable brain network markers of major depressive disorder across multiple imaging sites,” PLOS Biology, vol. 18, no. 12, pp. 1–26, Dec. 2020, doi: 10.1371/journal.pbio.3000966.

I would like to thank the authors for providing the highly detailed rebuttal. I have increased the rating from 6 to 8. Good job on the explanations and extra work.

Let us start by thanking the reviewer for his/her effort to assess our submission and the provided feedback. We enjoyed reading the concise summary and strength statements, which accurately highlight all our paper’s contributions. Please find below our responses to your concerns about weaknesses and additional questions.

Weakness: The clarity of presentation needs to be improved, with typos scattered throughout the paper: In Section 2.1, I would suggest using the notation (instead of Sym_n) to denote the space of n × n real symmetric matrices, to be consistent with the the notation S^n_++ for the space of real n × n symmetric and positive definite matrices. "For a more comprehensive proof, please consult Appendix ??." "In addition, we have use(d) the relaxed" Section 3.3, in the definition of formula, there is a redundant right parenthesis ")."

Our response: We are really sorry for submitting a manuscript with many typos. At the time of submission, we had to realize that we did not reserve enough time for editing and proofreading our submission. Thank you very much for reporting these typos back to us. In the meantime, we proofread extensively and improved spelling and grammar in the revised manuscript.

Question: The authors demonstrate the effectiveness of DeepGeoCCA on both EEG-fMRI and multiview EEG datasets, but it is not clear how the proposed framework would perform on other types of neuroimaging data, such as diffusion tensor imaging (DTI) or positron emission tomography (PET) data. Can the authors comment on the potential applicability of DeepGeoCCA to these types of data?

Our response: Absolutely! The framework proposed in this study can be applied to any paired views where one or both are SPD matrices, and the goal is to maximize geodesic correlation among their latent representations.

For other modalities of neuroimaging data, such as diffusion tensor imaging (DTI) or positron emission tomography (PET) data, we find it a very interesting exploration. While we are not experts in DTI and PET ourselves, through a literature search, we have found that many related studies have attempted to leverage the covariance matrix of DTI, making it a highly relevant potential application for our framework. For example, a three-dimensional diffusion tensor represents the covariance of the Brownian motion of water within a voxel and is, therefore, mandated to be a symmetric, positive-definite matrix. Therefore, DTI has been formulated in the SPD manifold and explored using geometric statistics as early as 2007 by P. Thomas Fletcher and Sarang Joshi [1].

It is essential to note that the two modalities must be "paired.” In our simultaneous EEG-fMRI task, the alignment is based on simultaneously collected timestamps, i.e., temporal dynamics. In our two-view EEG task, they are also aligned based on timestamps. For novel applications with DTI or PET data, it is necessary to identify tasks with paired views. In a clinical setting, the task could be disease diagnosis. Naturally, views associated with the same subject could be considered as paired. Consequently, the representations learned with our framework might even impact downstream, clinically relevant tasks in related domains.

Reference:

[1]. Fletcher, P. Thomas, and Sarang Joshi. "Riemannian geometry for the statistical analysis of diffusion tensor data." Signal Processing 87.2 (2007): 250-262.

We thank the reviewer for taking the time to provide a detailed review of our contribution. We appreciate the on-point summary and the thorough feedback. Thank you also for your account of our contributions and strengths, introducing additional application domains we did not consider before. Please find our detailed responses to the weaknesses and questions below.

Weakness 1: The motivation of using the geodesic relaxation (section 3.2) in intuitive a CCA setting. Why do the data points have to lie near the geodesics? In normal CCA or even deep CCA, we never have such constraints to encourage data points to align with the components. Component analysis is about finding directions that explain the data, not the other way around.

(Part 1): This is a fantastic concern. We apologize for failing to fully articulate some conceptual ideas behind our modeling approach in the submitted manuscript. In this response, we will clarify these thoughts, hoping they contribute to a better understanding of the overall conceptual framework of the paper. Moreover, as requested by this reviewer, we improved the clarity of the revised manuscript by emphasizing key concepts in section 3. Let us first provide some intuitions based on analogies with classical CCA in the next paragraph before we substantiate them in subsequent paragraphs.

Our modeling objective was twofold. Our first objective was to identify non-linear transformations for paired-view data so that the latent SPD representations have maximal geodesic correlation (i.e., similar to the primary CCA objective in linear spaces). Our second objective was to impose additional structure beyond symmetry and positive definiteness, namely, distribution along geodesics, which we promote with the $\varepsilon$-geodesic constraints. We think that the reviewer’s comment is primarily concerned about the second objective (i.e., why the latent representations should be near the geodesics). In our opinion, the second objective extends the linear projection constraint of standard CCA to our problem setting. Linear CCA typically obtains a closed-form solution by transforming the constrained optimization problem into an eigendecomposition problem. We propose a neural network solution to maximize geodesic correlation while using the $\varepsilon$ geodesic constraints to encourage the latent SPD representations to be mostly distributed along the geodesics.

In this study, the introduced approach originates from a fundamental concept within the field of Geometric Statistics, referred to as the geodesic subspace, initially introduced in principal geodesic analysis (PGA) and related works [1-3]. In the context of Riemannian geometry, a geodesic is a curve that locally represents the shortest path between points, acting as a generalization of a straight line. Moreover, the geodesic subspace can be viewed as an expansion of linear spaces from Euclidean to Riemannian geometry. The concept of the geodesic subspace is pivotal in PGA as the projection on this subspace preserves the Riemannian distance, representing the manifold-valued data's variance.

In a formal sense, a submanifold $H$ of manifold $M$ is considered geodesic at the point $p \in H$ if all geodesics of $N$ that pass through $p$ remain geodesic of $M$. A geodesic submanifold $H$ at the Fréchet mean $\mu$ is represented as the exponential mapping of the linear span of $r$ tangent vectors {$w_i$}$\_{i=1}^r$ $\in$ $\mathcal{T}_{\mu} M$

as $N = \exp_{\mu}$ $\big($span({$w_i$}\_{i=1}^r)$$\big).

In PGA, tangent vectors {$w_i$}$\_{i=1}^r$ are computed by sequentially maximizing the Riemannian distance between each data point and the Fréchet mean. This process captures the variance of the data at each step while progressively removing the influence of previously derived directional projections. This completes the manifold-value data's dimensionality reduction. In this context, $w_1$ can be analogous to the first principal component in principal component analysis (PCA). This work goes beyond finding the geodesic submanifold's tangent vectors {$w_i$}$\_{i=1}^r$, obtaining a novel low-dimensional data representation is crucial.

References:

[1]. Fletcher, P. T., and Joshi, S. (2007). Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Processing, 87(2), 250-262.

[2]. Fletcher, P. Thomas, et al. "Principal geodesic analysis for the study of nonlinear statistics of shape." IEEE transactions on medical imaging 23.8 (2004): 995-1005.

[3]. Thomas Fletcher, P. "Geodesic regression and the theory of least squares on Riemannian manifolds." International journal of computer vision 105 (2013): 171-185.