EpilepsyFM: Foundation Model for Learning Generalized Epileptic Representations from EEG and SEEG Signals

• Regarding the challenges, especially 2 and 3 on Page 2, this reviewer cannot find methodological solutions in the proposed framework to tackle them.
• The explanation for why EEG and SEEG data are processed simultaneously in a single model is insufficient. No reference or experiment demonstrates that using both EEG and SEEG together helps the Epilepsy-specific foundation model learn better representations, making the rationale for using both datasets unconvincing.
• Additionally, the method lacks techniques for handling the differences in EEG and SEEG data characteristics when processing them in a single model. EEG and SEEG are different types of signals with several distinctions, such as measurement locations and commonly used sampling rates (SEEG contains more high-frequency components than EEG [1]). These differences in signal characteristics should be considered during the training process. However, the proposed method does not adequately address this aspect.
• Furthermore, the chosen baselines are insufficient to demonstrate the proposed method's effectiveness fully. Specifically, the baseline LaBraM uses only EEG data, while BrainBERT uses only SEEG data. Therefore, a valid rationale for using each model as a baseline for tasks involving the other signal type is required, or an explanation of whether more suitable baselines exist should be provided.
• While the authors presented their experimental results for justification, the 3-channel group masking needs a more rigorous description or mechanistic explanation. Note that the results in Table 8 seem to have no significant difference among channel-number groupings.
• From a technical perspective, this reviewer finds no meaningful contribution to this work. Each module is taken from the previous work with no apparent justification. The experimental results show no significant difference, especially compared to LaBraM. Their performance difference on SEEG seems to result from using SEEG during training, i.e., an imbalance in the training datasets between models.

1. The proposal, including the VQ-tokenizer, the reconstruction loss in the first training phase, and mask prediction, is quite similar to LaBraM. What are the technical contributions of EpilepsyFM?
2. iEEG does not follow the standard 10-20 system, as it focuses on recording from specific, localized brain areas. How can such spatial information be generalized to different channels or brain regions?
3. The datasets used in this paper are re-extracted from TUH, which may not be sufficient to support a large model. I do not agree that using this dataset alone constitutes a significant contribution.
4. Authors claim to address the lack of consideration for seizure mechanisms by proposing channel set masking. While localizing the seizure onset zone might serve as a more effective marker, early abnormal electrical changes (patterns) are also very important for detection and contribute to the channel issue. I do not believe that simply applying masking can adequately investigate seizure mechanisms.
5. Why does pre-training not work well for predictions in Figure 5?

1. One of the main advantages of foundation models is their ability to perform well across multiple tasks, which seems to be underexplored in this paper. This raises some confusion about whether the motivation for developing a foundation model specifically for a single epilepsy-related scenario is sufficiently compelling. Could the authors clarify the rationale behind applying a foundation model in such a specialized setting？

2. While the model’s interpretability is briefly mentioned, a more in-depth analysis of the learned representations, particularly in relation to clinical insights, would significantly enhance its practical relevance. Expanding on interpretability could also increase the model’s value for clinical applications.

3. The paper lacks a discussion on the computational demands of EpilepsyFM, especially given the model's inherent complexity. Providing details on efficiency and possible optimization strategies would be valuable for assessing its practicality in clinical environments.

I will reconsider my assessment upon reviewing the rebuttal.

First of all, the writing of the paper should be improved. There are too many grammatical errors, e.g., in "For epilepsy, a clinical neurological disorder, we hope to break through the problem that deep learning can only solve a single task, ...."

Moreover, I doubt that the paper has been checked by an epileptologist, since there are statements that are simply incorrect. For instance: "Extracranial electroencephalography (EEG) and intracranial stereoelectroencephalography (SEEG) are crucial for epilepsy diagnosis." SEEG is not used for diagnosis but for surgical planning instead. SEEG requires implanting electrodes inside the brain, and is only done if the patient has been diagnosed with epilepsy. Another example: "Predicting epilepsy signals is essential for developing early warning systems for patients in need of preventive measures". The authors probably mean the prediction of seizures instead of "epilepsy signals".

Various foundation models for EEG have been proposed already, as mentioned in the paper. Moreover, recently a different foundation model for epileptic EEG has been proposed (mot mentioned in the paper): "Nested Deep Learning Model Towards a Foundation Model for Brain Signal Data", https://arxiv.org/html/2410.03191v2. The novelty of this submission is limited, as similar models have been proposed before. The novelty seems to lie in the fact that the model has been trained on EEG and SEEG. However, I dont understand the benefit of training a single model for EEG and SEEG, as both types of recordings are very different in terms of recording procedure and signal characteristics.

The model appears to be an extension of LABRAM without significant novel contributions. The improvements over LABRAM are marginal on key tasks, which raises concerns about the originality and necessity of the proposed approach. The metrics used (MAE,MSE) may not fully capture the clinical relevance of seizure prediction performance. While reconstructing the signal is useful, much of the problem lies in detecting the seizures, and the length of the seizures. The paper provides insufficient detail on the channel set masking approach. Specifically, the rationale for grouping channels into sets of three and how this impacts the model's learning is not well-explained, making it difficult to understand or reproduce the method. The channel masking strategy seems to focus on local channel clusters. Still, it does not account for long-range connections between brain regions, which are particularly important in SEEG data due to its extensive coverage of intracranial areas.