Do We Really Need Message Passing in Brain Network Modeling?

Q1.Reproducibility is a concern. Although the authors claim significant performance improvements over state-of-the-art models with a relatively simple architecture, they have not provided code access, making independent verification challenging.

R1. According to your suggestion, the source code has been made available at https://anonymous.4open.science/r/BQN-demo for verification purposes.

Q2. Some of the related work mentioned appears to be extraneous or not directly relevant to the scope of this study. While Graph Neural Networks (GNNs) are foundational models for brain network analysis, it is unclear which specific models, especially SGC and APPNP in Eq. 3, are being utilized in this context.

R2. We agree that the inclusion of certain related work, such as SGC and APPNP, may not be directly relevant to the scope of this study. While these models are foundational in the context of GNNs and their propagation mechanisms are indeed important for understanding message-passing models, their specific application in brain network analysis is limited. Therefore, we will adjust the manuscript to focus more directly on the models and methods that are most relevant to our study. Specifically, we will move the detailed discussion of SGC and APPNP to the appendix.

Q3.Although the meaning can be understood, the expressions "with" and "without" used in Figure 4 are not conventional. The authors are advised to adopt more standard notations, such as "BQN" and "w/o residual" for clarity. And in the caption of Figure 4, QBN is clearly a misspelling.

R3. Thanks for your careful review. We will revise the legend in Figure 4 to "BQN" and "w/o residual" as suggested. Additionally, we will conduct a thorough review of the manuscript to correct any misspellings, including the error in the caption of Figure 4.

Q4. The statement of matrix multiplication is inconsistent. For example, the notation $\mathbf{A} \cdot \mathbf{X}$ is used in Eq.3, while $\mathbf{Z} \cdot \mathbf{W}$ appears in Eq. 4.

R4. Thank you for pointing this out. We will revise the notation to maintain consistency. Specifically, we will update Eq. 4 to use the same matrix multiplication notation as Eq. 3, removing the symbol $\cdot$. This will ensure uniformity in our mathematical expressions.

Q1. Stacking multiple layers of Eq. 10 is theoretically equivalent to a single-layer formulation. The paper lacks theoretical or empirical justification for stacking multi-layer of Eq. 10 leads to performance gains.

R1. There may be a serious misunderstanding. Stacking multiple layers of Eq. 10 is NOT equivalent to a single-layer formulation. Take the case of a two-layer BQN as an example. Formulation $AW_A^1 \odot AW_A^2$ at the vector level is equivalent to $(aw_a^1) (aw_a^2)$, where $W_A^l$ means linear transformation of layer $l$. Subsequently, take the example of $a$ contains two variables, $(aw_a^1) (aw_a^2) = (a_1w_{a1}^1 + a_2w_{a2}^1) (a_1w_{a1}^2 + a_2w_{a2}^2)≠(a_1w_{1} + a_2w_{2})$ is clearly obtained for multiply interaction of variables $a_1$ and $a_2$. Thus, at the matrix level $AW_A^1 \odot AW_A^2≠AW_A$, which means stacking multiple layers of Eq. 10 is not equal to the single-layer formulation.

Besides, the performance gains of multi-layer come from the captured community structure in Eq. 11, which is the objective function of NMF-based community detection. Eq. 14 is the iterative updating rule of solving Eq. 11, and is equivalent to Eq. 10. Thus, multi-layers of Eq. 10 is the iterative updating rule of solving Eq. 11, which can learn better community structure and guarantee the performance gains.

Q2. More related studies and large datasets should be included.

R2. We have incorporated the mentioned methods into our effectiveness and efficiency comparison and expanded our evaluation to include two additional datasets (ADHD-200 and PPMI), more large-scale than ADNI. All methods use the same data preprocessing, brain network construction and uniform data partition.

No comparison was made with the work [3] for no open source. AGT requires datasets with more than two classes, limiting its evaluation on the PPMI dataset. The experimental results are shown in https://anonymous.4open.science/r/BQN-demo/figure/Table.jpg

Table reveals that BQN outperforms the baselines on the ABIDE, ADNI, and ADHD-200, demonstrating its superiority. On the PPMI dataset, BQN is not superior to AGT but still shows competitive performance while achieving the best efficiency. This highlights the promising potential of our model.

Q3. The reported results of ALTER and ContrastPool are different from original papers. ?

R3. Existing GNNs and Transformers for brain networks are very different in data processing and model selection. It makes it difficult to fairly assess their performance. To alleviate this difficulty, this paper unifies these settings and compares methods in a fair manner.

• model selection. We unify early stopping criteria as the lowest loss on the validation set. This is different from ALTER, which uses the criteria of the highest AUC. This causes the performance gap of ALTER. We also find that the proposed BQN also outperforms ALTER with the criteria of highest AUC.
• data processing. We unify data preprocessing and brain network construction as BioBGT[ICLR'25], ALTER, and BrainNETTF. This is different from the ContrastPool, which follows [5]. This causes the performance gap of ContrastPool. Existing work has demonstrated that different brain network construction methods can lead to different performance.

[5]Data-driven network neuroscience: On data collection and benchmark. NeurIPS, 2023

Q4. A convergence plot is expected to support a reliable trained model using the ADNI dataset.

R4. The convergence plot, which includes training loss, validation loss, and test accuracy using BQN, is available at https://anonymous.4open.science/r/BQN-demo/figure/BQN_convergence.jpg. The overall trend of the three experimental metrics is steady, despite some fluctuations. This indicate that the model generalizes well without severe overfitting.

Q5. The authors should clarify the chosen readout function theoretically aligns with BQN framework.

R5. We employ OCREAD, which concatenates the embeddings of cluster centers, as the readout function in our proposed method, consistent with BrainNETTF and ALTER. OCREAD exploits the clustering property for readout, which is consistent with the quadratic network in BQN as shown in Theorem 5.1.

Besides, traditional readout mechanisms, such as MEAN and MAX, remain valid for BQN, since they meet the requirements of readout mechanisms from node embeddings to graph embeddings. However, we believe their performance is not as good as OCREAD, since the clustering property is not considered.

Q6. Elaborate the transformation from the output of the model to brain graphs and provide thresholding details in the conversion.

R6. Given the output of the final BQN layer $A$, it is first calculated as $A=(A+A^T)/2$. Next, $A_{Template}^{ASD}$, $A_{Template}^{NC}$ and $A_{contrast}$ are obtained by equations in Lines 423, 424 and 428 repectively. Finally, the top-20 edges with the largest weights from $A_{contrast}$ are selected for visualization.

Q1.The parameters b and c in Eq. 7 are not represented or utilized in BQN(Eq. 8). Should the layers of the model use MLP?

R1. Yes, the matrix $\mathbf{W}$ in Eq. 8 represents a Multi-Layer Perceptron (MLP), and the parameters $b$ and $c$ in Eq. 7 are realized in the model implementation by setting bias=True for the MLPs.

Q2.The results are somewhat insufficient. Given the limited number of datasets used and the lack of consistent trends across multiple metrics, it is recommended that the authors provide additional experimental results for other metrics in Figures 2, 3, and 5.

R2.According to your suggestion, we have conducted additional experiments to supplement the results in Figures 2, 3, and 5. For Figures 2 and 3, we have incorporated precision, recall and micro-F1 metrics on experiments with the ABIDE and ADNI datasets. The results are available at the following links: Fig2_ABIDE, Fig2_ADNI, Fig3_F1. These supplementary experiments demonstrate consistent trends across the datasets. For Figure 5, we utilze the micro-F1 score metric to provide a more comprehensive evaluation of model performance as the number of layers increases. The results can be accessed at Fig5_F1. We believe these additional results enhance the robustness of our findings and address the concerns regarding the sufficiency of the experimental evidence.

Q3.Some punctuations are missing after certain formulas, such as Equations 6 and 8. In Equation 8, it seems that $W_A$ is reused

R3. Thank you for pointing this out. We will carefully review the manuscript to ensure proper punctuation is used throughout, especially after equations. We will revise Equations 6 to:

And for Equation 8, we will revise it to:

For $\mathbf{W}_A^l$ is reused, we use $\mathbf{W}_A^l$, $\mathbf{W}_B^l$ and $\mathbf{W}_C^l$ instead. And we will check variable symbols throughly to correct any reuse errors.

Q1. The model analysis on GNN and transformer provides evidence to question the message passing in brain modeling. The proposed BQN makes sense by theoretical analysis on its clustering property. Its rationality is also guaranteed by the theory progress in the Quadratic Network. I suggest that the authors include them in the manuscript and emphasizing model efficiency in the introduction and abstract.

R1. Thanks for your suggestion. We will incorporate the mentioned contents of the Quadratic Network into the manuscript, referencing [1], [2], [3], and [4]. Additionally, we will emphasize the model efficiency in both the introduction and abstract to better highlight the contributions of this paper.

[1]Universal approximation with quadratic deep networks. Neural Networks, 2020

[2]An attention free transformer. arXiv preprint, 2021

[3]Attention-embedded quadratic network (qttention) for effective and interpretable bearing fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 2023

[4]Towards efficient and interpretative rolling bearing fault diagnosis via quadratic neural network With Bi-LSTM. IEEE Internet of Things Journal, 2024.

Q2. The source code is not provided. I suggest including it in the supplementary material.

R2. According to your suggestion, the source code has been made available at https://anonymous.4open.science/r/BQN-demo for verification purposes.

Q3. The experimental evidence should be provided to justify its clustering property.

R3. In response to your suggestion, we have analyzed the clustering properties of the proposed BQN model. The brain is segmented into functional regions using established criteria from prior research [5][6]. The clustering performance is evaluated using three standard metrics: Silhouette Coefficient (SC), Calinski-Harabasz Index (CH), and Davies-Bouldin Index (DB). Results are obtained for both the original data and the outputs of three well trained BQN models, using 1 layer, 2 layers and 3 layers respectively.

The results reveal that the proposed BQN effectively captures the clustering properties of functional brain regions.

[5]Distinct brain networks for adaptive and stable task control in humans. Proceedings of the National Academy of Sciences, 2007

[6]Prediction of individual brain maturity using fMRI. Science, 2010

Q4. There are some methods based on prototypes, such as [Kan et al., 2022b], the essence of which is clustering. Why does BQN outperform them?

R4. The performance superiority of the proposed BQN is primarily due to its quadratic network’s approximation capabilities and robust model architecture.

Firstly, BrainNETTF[Kan et al., 2022b] use the Pearson matrix as the feature matrix, which already captures holistic brain region interactions. This limits the capacity of the Transformer's fully connected graph messaging mechanism. In contrast, BQN learns clustering properties and employs a quadratic network with more general function approximation capabilities, resulting in better ROI and graph representations. Secondly, BrainNETTF is a Transformer-based model with larger parameters, increasing the risk of overfitting on relative small brain datasets. BQN, with fewer parameters, reduces model complexity and inference uncertainty.