Long-range Brain Graph Transformer

We thank you for your detailed feedback and your questions. We hope we have well addressed your concerns. If there are any other issues remaining, we are pleased to address them further.

W1.a In fact, we employ Pearson correlations as adaptive factors in random walk, without any modifcation. Since the correlation between ROIs reflects their communication strength, which is crucial for comprehending the functional organization, dysfunctions, and information propagation. We take the correlation between ROIs as adaptive factors to influence the exploration mechanism of random walk, which induces ROIs with stronger connectivities to exhibit higher transfer probabilities of the next hop. This manner simulates real-world brain-wide communication.

We recognize that this part of the presentation may not have been clearly articulated, leading to potential misunderstandings. Therefore, we have make the necessary revisions in the revised paper to enhance clarity.

W1.b The introduction of degree matrix can help to obtain richer information about brain-wide communication and is very commonly used in network analytics [1]. Since the degree matrix provides the number of degrees for each ROI, its ability to reflect the active state of the ROI in communication is important in determining which ROIs play a key role in information propagation. Hence, a degree matrix determines the transfer probability of a node to its neighboring nodes and highly influences the behavior of walker [2]. We have now incorporated these details in the revised paper.

W2 We apologize for our mistake, we did mix up $k$ and $K$. As you say, what we use is $k = K$. $k$ is generally denoted as the number of hops in random walks. We apologize for not defining it explicitly, which has now been addressed in the revised paper.

W3&Q1 While nearly all of the experimental settings are presented in the source code, we acknowledge that our descriptions of these settings were indeed insufficient. We have now incorporated the details in Appendix A.

Specific details requested by the reviewer includes:

• For the ABIDE dataset, we use the Craddock 200 atlas with 200 ROIs. For the ADNI dataset, we use the AAL atlas with 90 cortical ROIs and 26 cerebellar ROIs.
• The threshold is 0.3.
• This is a mistake in the number of hops reported in the paper. The experiments were performed with 16 hops. We tried the hops commonly used for long-range dependency capture in network analysis by random walks through a grid search, and chose 16 hops. As shown in Figure 4(a) in the paper, larger hops do not lead to better results since brain graphs are relatively dense.

W4&W5 Upon receiving comments, we promptly review the referenced papers and their corresponding open-source codes. As shown in Tables 1&2&3 of the global response, we have added additional datasets (PPMI, Matai, TaoWu, and Neurocon) and baselines (A-GCL [3] and ContrastPool [4]).

Regarding the complete ADNI dataset [5], we discovered from the GitHub repository provided by the authors that due to data protocol issues, it has not yet been made publicly available. Regarding [6], we find that the original papers did not provide corresponding code links, and also not released in github. We also contacted the authors, but as of the end of the rebuttal, we have not received responses. However, we need to state that [6] more focuses on learning time series of fMRI using a global and local matching-based graph kernels to construct dynamic brain networks, whereas ALTER is more concerned with capturing long-distance dependencies from static brain networks using random walk kernel, which are fundamentally different from each other.

In the field of neuroscience, it is often challenging to obtain large amounts of data due to various constraints. And often use a smaller dataset (≤130 individuals), which are sufficient to support conclusions [7, 8, 9, 10]. We initially focused only on the ADNI and ABIDE because they are widely used for disease prediction. However, due to the significant challenges in preprocessing, we were unable to process the complete fMRI data from the ADNI dataset. While only two classes and 130 subjects were included, our method still shows excellent performance. This demonstrates the effectiveness of ALTER with limited data, suggesting that it may better adapt to other datasets with similarly limited quantities. This adaptability can potentially lead to more successful outcomes in other brain science-related medical tasks.

W6 As shown in the Table 5 of the global response we did additional experiments and achieved better performance. This is because ALGA can capture the long-range dependencies in the brain network.

W7&Q2 Based on the SHAP model, we have supplemented the experiment in neuroscience perspective, and concluded that ALTER is can recognize disease-related regions and give high attention. The results are illustrated in Figure 1(a) of the global response.

References

[1] Graph neural networks with learnable structural and positional representations. In ICLR 2022.

[2] How to Count Triangles, without Seeing the Whole Graph. In KDD 2020.

[3] A-GCL: Adversarial graph contrastive learning for fMRI analysis to diagnose neurodevelopmental disorders. MIA 2023.

[4] Contrastive graph pooling for explainable classification of brain networks. TMI 2024.

[5] Data-driven network neuroscience: On data collection and benchmark. In NeurIPS 2023.

[6] Effective graph kernels for evolving functional brain networks. In WSDM 2023.

[7] Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network Classification TMI 2024.

[8] RH-BrainFS: Regional Heterogeneous Multimodal Brain Networks Fusion Strategy. In NeurIPS 2023.

[9] Functional brain network reconfiguration during learning in a dynamic environment. Nature Communications 2020.

[10] Increased global integration in the brain after psilocybin therapy for depression. Nature Medicine 2022.

We thank you for acknowledging the novelty of the proposed method and for suggesting relevant analysis, which we have included in the global response. We hope to provide satisfying answers to the concerns raised. If there are any other issues remaining, we are pleased to address them further.

W1 Our model aims to capture long-range dependencies within brain networks leveraging the biased random walk, which has a Markovian nature that captures long-range communication among brain ROIs by sampling and encoding sequences into embeddings. We agree that random walk is common in graph representation learning. However, thedifferent pairs of ROIs in brain networks usually exhibit different communication strengths in brain activity. As a result, traditional random walk methods are usually not applicable to brain networks, and cannot capture long-range dependencies in brain networks.

W2 We acknowledge that Transformer is commonly used in brain network analysis. The focus of the proposed method is on capturing long-range dependencies within brain networks. Transformer inherently has limitations in capturing long-range dependencies, which prevents a comprehensive understanding of brain-wide communication. Instead, the proposed ALTER method introduces an adaptive long-range aware strategy to explicitly capture long-range dependencies within brain networks. It then integrates long-range and short-range dependencies using the adaptive mechanism of Transformer, thereby achieving a multi-layered understanding of brain networks.

W3 As shown in Figure 1(a) of the global response we supplemented the interpretability analysis.

Q1 In the submitted manuscript, we evaluated performance solely on binary classification tasks using the ADNI and ABIDE datasets. To further demonstrate the effectiveness of the proposed method, we have conducted additional experiments on the PPMI dataset with four class, as suggested by reviewer s7Az. The experimental results are presented in the Table 1 of the global response. The results show that the proposed ALTER method achieved the best performance, as it effectively captures long-distance dependencies between brain ROIs.

Q2 Thank you very much for your question. Since we concatenate the long-range dependency embeddings directly with the original feature embeddings without any other operation, the dimensionality of the concatenated features is not equal to that of the original feature embeddings. We have now revised the paper to improve the clarity of the methodology.

Q3 Thank you very much for your question. The value of the biased random walk strategy in ALTER for multi-modal data is indeed a topic worth exploring. However, due to some fundamental differences between functional and structural brain networks constructed based on fMRI and DTI, directly applying the biased random walk strategy to capture long-range dependencies in them may not be feasible and would require tailoring the ALTER model. Nonetheless, the biased random walk strategy can be adjusted by learning consistent and complementary communication patterns between functional and structural brain networks, which may enhance its adaptability to multimodal brain network data. We have now incorporated this as a future work of the study in the revised paper.

We thank you for your detailed assessment of our work and for highlighting the merits of our approach, as well as the importance of the problem. We address all concerns below, if there are any other issues remaining, we are pleased to address them further:

W1 Brain networks are inherently dense, which has led to the increasing use of transformers for their analysis. However, a significant limitation of transformers is their inability to effectively capture long-range dependencies, which limits the integrated understanding of brain-wide communication. The proposed ALTER is concerned with the capture of long-range dependencies between ROIs, not the combination of random walk and graph transformer. In particular, we design the ALGA strategy, which simulates real-world brain-wide communication by utilizing adaptive factors to evaluate the communication strength between ROIs and capturing long-range dependencies in brain networks through a Markov process. We have now re-emphasized the novelty and difficulty of the proposed ALTER in the revised manuscript.

W2 We acknowledge that the proposed method exhibits a lower SPE on the ADNI dataset compared to BrainNETTF and a lower SEN on the ABIDE dataset compared to Graphormer. However, considering the standard deviation, our method demonstrates more stability, which can be attributed to its ability to capture long-range dependencies within the brain network. Moreover, our method significantly outperforms the baselines on other metrics across both datasets. In particular, on the ADNI dataset, our method shows a 10.3% improvement in the SEN metric compared to the sub-optimal method. We have provided a detailed explanation of this aspect in the revised manuscript.

W3 Thank you for requesting clarity on the linear layer. The main objective of introducing a linear layer to map the initial long-range embedding to the transformer is to enable end-to-end learning of the long-range embedding to participate in updates and enhance its expressiveness. This method is able to enhance the expressiveness of the random walk embedding by capturing multiple types of graph propagation, which in turn facilitates the integration of global information in the brain graph. We have now added these details in the revised paper. If we do not use a linear layer and directly concatenate the initial long-distance embedding with the original features, this reduces the expressiveness of the long-distance dependencies in the brain graph learning. We additionally performed ablation experiments on ABIDE dataset to demonstrate the effectiveness of introducing a linear layer.

Q1 we utilize the proposed ALTER method to qualify these dependencies as they manifest in the data. In brain network analysis, long-range dependence and short-range dependence refer to interactions between neurons in the spatial dimension. Among them, short-range dependencies usually refer to interactions that occur in the same brain region or between anatomically neighboring brain regions. In contrast, long-distance dependence usually spans larger spatial brain regions and involves the information transmission between different functional regions. Such long-range dependencies assume the integration of global information, which plays a key role in complex cognitive functions. In the manuscript, we only deal with the capture of long-range dependence without defining it explicitly at the neuroscience level.

Q2 Indeed, we have not yet explored other correlation measures besides the Pearson correlation coefficient to define the adaptive factors in our study. The proposed method represents an initial attempt to capture long-range dependencies in brain networks. In future work, we plan to investigate some non-linear correlation measures, to further refine and potentially improve the adaptability of our method. It is worth noting that studies in neuroscience predominantly rely on Pearson correlation.

Q3 In ALTER, the self-transformer mechanism is introduced to adaptively integrate the long-range and short-range dependencies between ROIs. Meanwhile, at the suggestion of reviewer s7Az, we try to replace the transformer with 1-layer GCN and 2-layer GCN. The experimental results are shown in the Table 5, which it is proved that transformer usually achieves better results due to its ability of adaptive learning.

Q4 In Figure 5 we show additional examples of long-range capture at the individual level on the ABIDE dataset. In Figure 5(a)(b)(c) we can see that despite the fact that the two red labeled nodes are 5 hops apart, there is still a high attention value between the two ROIs. In Figure 5(d) we can see that there is a high attention value between 6 and 12 which are 6 hops apart.

We thank you for the comments on our work. Below we address the questions raised. We hope we have well addressed your concerns. If there are any other issues remaining, we are pleased to address them further.

W1 As shown in Figure 1(a) of the global response, we have analyzed the output using the SHAP model and confirmed that our results reflect AN-related ROIs. The proposed ALTER promotes an integrated understanding of brain-wide communication by capturing long-range dependencies and achieves superior performance in disease prediction tasks. Long-range dependencies, as complementary to short-range dependencies, can deliver unique insights into the organization and behavior of brain networks associated with neuropsychiatric disorders [1, 2, 3]. ALTER effectively captures long-range dependencies using the Markov process-based ALGA strategy, which has the ability to further explain the important disease-specific pattern and find the biomarkers. It is worth noting that the proposed model employs Transformers which are very well known for extracting the important features using its attention mechanism, which is effectively used for interpretability.

W2 We apologize for not providing a clear description of how to implement the comparable baselines and build the brain graphs for them. We have added a section in the appendix to clarify the implementation details of the comparable baselines and the construction of the brain graphs.

Specifically, to ensure a fair comparison, we use the open-source codes of BrainGNN, BrainNETTF, and FBNETGEN. For SAN, Graphormer, LRGNN, and GraphTrans, we adapt their open-source codes and modify them to suit the brain network datasets. For BrainNetGNN, we implemente it ourselves following the settings described in the paper. During the parameter tuning, we follow the tuning of BrainNETTF [4] for SAN, BrainGNN, FBNETGEN, Graphormer, and BrainNETTF. For BrainNetGNN, we search the number of GRU layers {1, 2, 3}. For LRGNN, we vary the aggregation operations {8, 12} with the number of cell {1, 3}. For GraphTrans, we search the number of GNN layers {1, 2, 3, 4} with the hidden dimension of 100.

We utilized functional connectivity matrix to compute a brain graph for BrainNETTF, which is computed by calculating the correlation between brain regions using the processed fMRI. The details of computing these correlation matrices is also incorporated to the revised paper. For BrainNetGNN and FBNETGEN, the models required the processed fMRI as input. BrainGNN, SAN, Graphormer, LRGNN, and GraphTrans required the correlation matrix and adjacency matrix. As mentioned in the paper, the adjacency matrix is obtained by thresholding (≥0.3) the correlation matrix.

W3 We thank the reviewer for requesting a clear explanation of the preprocessing steps for fMRI. We preprocess the fMRI using the Data Processing Assistant for Resting-State Function (DPARSF) MRI toolkit. Specifically, we removed the first 10 time points from the downloaded nii data according to the default mode and chose slice timing, where the middle layer was the reference slice. meanwhile, we set the head motion correction to ‘Friston 24’, and selected automask and Nuisance covariates regression. the others were basically set according to the default mode. Then, considering individual differences, we choose to perform ‘Normalize by DARTEL’, and for the definition of ROIs, we adopt the altas already available in DPARSF. Then, we construct brain networks $G = \left( {V,X,A} \right)$ for each fMRI.

During the experiment, for the ABIDE dataset, since we directly adopted the processed brain network from [4] and used it as the correlation matrix, its quality control refers to [4]. For the ADNI dataset, besides the above preprocessing, we performed head motion correction, slice timing correction, realigning and normalize. Please note that we have followed the standard protocol used by the other research studies to ensure that any bias and noise have been removed from the dataset.

Q1 The final loss function for this task is the cross-entropy loss as the model address a classification task. This is already implemented in our open-source code as shown. We have now revised the paper to improve the clarity of the methodology.

Q2 For all the experiments, we repeated them 10 times. Instead of k-fold cross-validation, we conducted repeated random split validation. To ensure that our results are trustworthy, we additionally performed 5-fold cross-validation (as shown in Table 4 in the global response).

Q3 We have computed the average across individuals to perform group-level analysis, as this approach aligns with the methodologies commonly adopted in similar studies [5]. The average graph and the corresponding attention heatmap are illustrated in Figure 1(b)&(c) of the global response. We can observe that ALTER captures group-level long-distance dependence, but it is not very significant relative to the individual-level. This may be due to certain individual differences in patients, including age and gender, which can affect the brain-wide communication [6].

References

[1] Space-independent community and hub structure of functional brain networks. NeuroImage 2020.

[2] Long-range connections are more severely damaged and relevant for cognition in multiple sclerosis. Brain 2019.

[3] Engineering brain assembloids to interrogate human neural circuits. Nature Protocols 2022.

[4] Brain network transformer. In NeurIPS 2022.

[5] Structure-function coupling in the human connectome: A machine learning approach. NeuroImage 2021.

[6] Local structure-function relationships in human brain networks across the lifespan. Nature 2022.