Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG

Thank you for all your comments. We respond to your comments one by one as below.

---

🟩 Q1: Claims and Evidence

(1)
We understand your concerns; however, we believe they stem from a misunderstanding of our approach. Our method is purely based on functional connectivity EEG graphs, rather than distance-based spatial EEG graphs.

Graphs based solely on spatial distances lead to spatial bias: positive pairs are close while negatives are far apart. For instance, Ho et al. (AAAI 2023) used hybrid graphs combining spatial and functional connectivity. Though helpful, this introduces spatial bias in contrastive learning.

Our results (see the Table in Q4 response to Reviewer c7XS) show that spatial graphs degrade performance even with our distillation. Hybrid graphs offer slight improvements, but spatial bias remains.

Thus, we use only functional connectivity—nodes use PSD features, edges are defined by Pearson correlation—to avoid spatial bias. This ensures positive/negative pairs are based on functional, not spatial, similarity.

(2)
In EEG, distant electrodes often show strong functional connectivity. Our graph is built on global functional correlation (Pearson), not spatial proximity—tailored for resting-state EEG.

Even electrodes that are spatially far apart can exhibit strong connections when computing functional connectivity, which we define as positive sample pairs. Functional connectivity inherently ignores spatial distance. Therefore, the concern you raised regarding spatial locality is mainly relevant to distance-based spatial connectivity, not to functionally derived connectivity.

Our method captures global patterns without spatial constraints.

---

🟦 Q2: Methods and Evaluation Criteria

(1) Lack of Strict Subject-Independent Setup
As mentioned in the Q1 table for Reviewer c7XS and shown in the Table below, we conducted both subject-dependent and subject-independent experiments on clinical and SEED datasets. The model was pre-trained only on clinical EEG and fine-tuned on SEED to assess generalization.

Table. Performance on Sex Classification (EMBARC) and Emotion Recognition (SEED)

(2) Missing Comparison to Non-GNN Pre-training

We compared: (1) contrastive pre-trained GNNs (GraphCL), (2) masked-reconstruction pre-trained GNNs (GraphMAE), (3) masked-reconstruction pre-trained time-series models (LaBraM), and (4) our joint contrastive + reconstruction GNN. Our model outperforms prior pre-trained GNNs and slightly surpasses time series-based pre-trained model LaBraM, while being significantly more efficient.

Note that although LaBraM performs well, it demands far more memory and parameters.

(3) Underwhelming Gains

1. Performance gains on clinical resting-state EEG are modest, but on SEED (emotion recognition), we observed >10% improvement post fine-tuning—much higher than in disease classification.
2. Our pre-training data is limited; larger datasets and augmentation will likely enhance gains.
3. Our goal is boosting low-density EEG performance with lightweight models. Our method enables small LD models to match HD models, showing substantial relative improvements.

---

🟨 Q3: Other Comments or Suggestions

Thank you for the suggestion. Following advice from Reviewer c7XS and FWdZ, we simplified notations, removed redundant symbols from the text and figures, and deleted the symbol table from the appendix. This significantly improves clarity and readability.

Thank you for all your comments. We respond to your comments one by one as below.

---

🟩 Q1: Methods and Evaluation Criteria

(1)
Our pre-training framework and distillation loss are designed to be general and compatible with both major types of GNNs: local message-passing models (e.g., DGCNN) and global attention-based models (e.g., graph transformers). Both our graph contrastive learning (GCL) and graph masked autoencoding (GMAE) methods work across different GNN architectures, as both types can capture global graph topology from different perspectives.

The goal of both pre-training and distillation is to learn meaningful global structures, regardless of the underlying GNN. The choice of backbone affects performance due to their distinct inductive biases: local GNNs excel at capturing neighborhood-level patterns, while graph transformers are more effective at modeling global interactions.

In practice, we recommend using message-passing GNNs like DGCNN for small graphs due to their ability to capture fine-grained local structures, and graph transformers for large graphs, as they offer better scalability and efficiency.

(2)
Our Graph Topology Distillation (GTD) loss defines positive and negative pairs based on functional similarity in EEG graphs. For more theoretical analysis, please refer to (Joshi et al., 2022. TKDE 2022).

In EEG, graph connectivity reflects functional correlations rather than spatial proximity, so strong links can exist between spatially distant electrodes. However, in low-density (LD) graphs, missing electrodes may break these meaningful connections. For example, two strongly connected nodes in the HD graph may become disconnected in the LD graph due to missing intermediaries. These lost but meaningful links are treated as positive pairs in our distillation.

Conversely, missing electrodes can also lead to spurious connections in the LD graph that are not present in the HD graph. These are considered negative pairs. The distillation objective encourages the LD model to approximate the topology of the HD graph by explicitly distinguishing such positive and negative connections.

---

🟦 Q2: Experimental Designs or Analyses

(1)
Thank you for the suggestion. We use mean squared error (MSE) to evaluate reconstruction quality.
The MSE losses for the four cases in Figure 4 (b), (c), (d), and (e) are 0.25, 0.31, 0.44, and 0.17, respectively. These values align well with the visual quality of the reconstructions, further supporting the effectiveness of our approach.

(2)
Random selection is not applicable, as reducing high-density (HD) EEG to low-density (LD) EEG follows specific electrode selection rules.
However, to assess model robustness, we simulate extreme conditions by randomly dropping electrodes in multiple trials. Given the same number of remaining electrodes, performance with random drops is generally worse than with structured downsampling based on predefined rules or distributions.

---

🟨 Q3: Other Strengths and Weaknesses

(1)
We sincerely appreciate your recognition and valuable suggestions. In response, we have revised Figure 1 by removing redundant symbols and replacing them with clearer textual descriptions. This change has notably improved the clarity and readability of the model illustration.

(2)
We have added the following analysis of Table 4 in the revised manuscript:
We compared our proposed GTD loss with several commonly used graph distillation losses. As shown in Table 4, GTD consistently outperforms the others. Moreover, combining GTD with traditional logits distillation achieves the best performance, as it allows the model to distill both semantic information from logits and structural information from the graph topology.

---

🟪 Q4: Questions for Authors

(1)
Yes, we incorporated multiple frequency bands as input features and observed a noticeable performance improvement.

(2)
Yes, as shown in the Table of the answer of Q1 for Reviewer c7XS, we evaluated our model on the emotion recognition task using the SEED dataset. Despite being pre-trained on a medical resting-state EEG dataset, our model can still be effectively fine-tuned for the emotion recognition task. We expect that pre-training on a task-specific EEG dataset (e.g., emotion recognition) would further enhance the model's performance.

(3)
No, we have not explicitly explored using low-dimensional data to reconstruct high-dimensional (i.e., more-channel) data as a form of data augmentation or upsampling. However, this is indeed an interesting direction. Leveraging the reconstruction ability of the encoder to infer additional channels could potentially enhance the representation capacity and improve downstream performance, but it is more difficult and needs more data to pre-train the generative model. We consider this a valuable future work direction if we have more data to do this.

Thank you for all your comments. We address each comment individually below.

---

🟩 Q1: Claims And Evidence: (Generalization to Emotion Recognition)

To further evaluate the generalization ability of our model, we conducted additional experiments on the SEED dataset, a widely used benchmark for emotion recognition. The results show that our model, pre-trained on resting-state EEG data, can be effectively fine-tuned for this task.

It is worth mentioning that, due to time constraints in collecting non-clinical resting-state EEG (e.g., task-related EEG from BCI domains), we used our medical resting-state EEG dataset for pre-training before fine-tuning on SEED. The EMBARC task is conducted on HD to LD distillation, and for the emotion recognition task, we also adopt HD(64 electrodes) to LD(32 electrodes) distillation.

Table 1. Performance on Sex Classification (EMBARC) and Emotion Recognition (SEED)

---

🟦 Q2:Claims And Evidence: (Model Design Motivation)

DGCNN and Graph Transformer are two representative types of graph neural networks (GNNs). DGCNN is a message-passing-based GNN, while the Graph Transformer is a spatial-attention-based GNN.

In terms of performance:

• GNNs like DGCNN are relatively lightweight but tend to achieve lower accuracy.
• In contrast, Graph Transformers generally yield better performance, albeit with higher model complexity and computational cost.

---

🟨 Q3: Methods And Evaluation Criteria: (Robustness to Perturbations and Variability)

We introduced noise into EEG signals and randomly dropped electrodes to assess model robustness. As shown in the table below, our model shows the highest resilience, with the smallest performance drop under noisy and incomplete inputs.

We also evaluated subject variability by comparing performance in subject-dependent and subject-independent settings. Our model demonstrates greater stability, showing the least degradation in the subject-independent scenario compared to other methods.

Table 2. Model Robustness under Perturbations

---

🟪 Q4: Methods And Evaluation Criteria: (Ablation on EEG Graph Construction Strategies)

We conducted an ablation study on different EEG graph construction strategies. As shown in the table below:

• Coherence and Mutual Information yielded the best performance.
• Pearson Correlation ranked next.
• Spatial Distance-based graphs performed the worst.

We speculate that spatial graphs may lead to overemphasis on spatial locality—as pointed out by Reviewer v1Sd—resulting in degraded performance. In contrast, functional or statistical connectivity better captures neural relationships and leads to improved results.

Table 3. Accuracy with Different EEG Graph Construction Methods