We thank the Reviewer for highlighting the compression performance of BrainCodec and acknowledging the concrete challenge represented by neurophysiological signals.

We have updated the manuscript to include more evaluation metrics and additional ablations that better characterize the model's performance.

---

W1. Evaluation metrics

We choose three separate evaluation metrics to better characterize the performance of BrainCodec:

1. Percent-root-mean-square-difference (PRD). PRD is the most used metric to determine the reconstruction fidelity of compressors on EEG and iEEG, as seen in the literature [1,2,3].
2. Downstream classification performance. Classification tasks are a concrete benchmark for the performance of BrainCodec, as they provide an immediately parsable metric of the distortion caused by the compression.
3. Subjective expert evaluation. The specific characteristics of EEG and iEEG are not as easily parsed as music or speech. Therefore, we assess the reconstruction fidelity through the expert evaluation of a neurologist who works with this data every day and is best poised to capture the fine difference between the original and the compressed signal.

Following the Reviewers' suggestion, we now include four additional metrics borrowed from the signal processing domain, namely:

1. PRD of the spectrogram (PRD-spec)
2. Root-mean-square error (RMSE)
3. Signal-to-noise ratio (SNR)
4. Peak signal-to-noise ratio (PSNR)

All tables in Appendix D have been updated accordingly, showing that BrainCodec's high reconstruction fidelity translates to these four additional evaluation metrics as well.

Briefly here as an example, we report all the metrics collected for the Base model trained and tested on the iEEG SWEC dataset.

[1] Higgins, Garry, et al. "EEG compression using JPEG2000: How much loss is too much?." 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE, 2010.

[2] Du, XiuLi, et al. "Fast reconstruction of EEG signal compression sensing based on deep learning." Scientific Reports 14.1 (2024): 5087.

[3] Nguyen, Binh, et al. "Wavelet transform and adaptive arithmetic coding techniques for EEG lossy compression." 2017 international joint conference on neural networks (IJCNN). IEEE, 2017.

---

W2. Additional ablations

We strive to validate BrainCodec in many scenarios to improve its real-world applicability. As suggested by the Reviewer we have now included ablations on the following:

1. Number of residuals. We assess the performance at varying compression ratios by varying RVQ's number of residuals from 1 to 16 (i.e., 16 to 256 compression ratio). For our main results, we vary the Encoder framerate instead, because we find it yields superior results. For example, at 16 compression ratio we report a PRD of 4.60 against 8.74, a notable improvement.
2. Codeword size. We vary the size of each codeword from 64 to 1024 (i.e., a compression ratio of ~51 to ~85). This gives diminishing returns, with our choice of 256 a consistent middle-ground.
3. Line length loss. We train a model without the line length loss. We find that the performance is decreased, with a PRD of 15.95 against 15.76. However, the difference is especially noticeable in the reconstruction of higher frequencies in the spectrogram, as reported in Appendix B.4.
4. Kernel size. We vary the initial kernel size from 3 to 7, and show that our choice (3) yields the best results.
5. Number of patients. We train additional models using 2 and 4 training patients. The results once again point to diminishing returns, indicating that our single-patient training already produces high-fidelity reconstructions, as highlighted by the Reviewer.

We report the most relevant ablations here. First, varying the number of residuals yields worse performance with respect to varying the Encoder framerate.

Second, the line length loss improves PRD, but most importantly improves the reconstruction of higher frequencies, as seen in Appendix B.4.

Overall, our results show that our choices of modules and objectives --- consistent with the characteristics of EEG and iEEG signals --- improve the reconstruction fidelity of BrainCodec and strengthen its effectiveness.

Reviewer HbiL

We thank the Reviewer for acknowledging the impact of our findings regarding the importance of data quality in the development of EEG foundational models.

We have updated the manuscript to include new ablations on the RVQ component, a new iEEG dataset (Brain Treebank), and an ablation with respect to the reference schema.

---

W1. RVQ ablations

We have chosen to vary the compression ratio based on the framerate of the Encoder because it yielded better performance and flexibility during our initial evaluation.

We now perform a systematic assessment (included in Appendix B.3), in which we vary both the number of residuals from 1 to 16 (i.e., a compression ratio of 16 to 256), and the size of each code-word from 64 to 1024 (i.e., a compression ratio of ~51 to ~85).

The performance at equivalent compression is notably inferior to our choice of varying the framerate.

---

W2. Quantized embeddings

Inspired by the promising results of VQ-based models we also tried to use our embeddings for the seizure classification task. Unfortunately, an initial evaluation yielded unsatisfactory results.

However, we are currently working towards fully integrating BrainCodec into this new and exciting area of research, but unfortunately, such immediate efforts are outside the scope of this manuscript.

---

W3. Additional iEEG datasets

We wish to remark that, as the Reviewer correctly pointed out in their Summary, we evaluate two iEEG datasets, the SWEC and the MC iEEG dataset, for a total of 106 subjects.

We are also always excited to find new high-quality datasets to include in our benchmarks. Following the Reviewer's suggestion, we have found the Brain Treebank dataset to be highly well-curated and now include it in our results, bringing us to a total of 3 iEEG datasets and 3 EEG datasets. Unfortunately, timing and resource constraints prevent us from also evaluating the Du-IN dataset.

We present the performance of our BrainCodec trained on the SWEC dataset in Appendices C and D. The results are comparable to testing on the SWEC dataset itself and show once more that our model generalizes well to new datasets.

Moreover, as requested by the Reviewer we also train a model on both SWEC and TreeBank datasets jointly and evaluate it on both the SWEC and TreeBank datasets separately to compare it with other models. The evaluation of this model has not been completed yet, but we commit to including it as soon as possible.

---

W4. Reference schema ablation

We thank the Reviewer for pointing us towards the un-processed Brain Treebank dataset, which allows us to also evaluate the effects of post-processing on our compressor.

We now include in Appendix B.1 the reconstruction results of the Brain Treebank dataset for a median, bipolar, and Laplacian (referenced to the mean of the electrode group) reference, and also no reference schema.

All results are rather similar, with median and bipolar referencing performing the best. However, the effect of the reference schema is mostly influenced by the collection of the dataset itself, rather than the compressor architecture. Overall, the reference schema does not notably influence the reconstruction fidelity of BrainCodec.

---

W5. BrainBert tasks

We prioritized running additional experiments such as ablations, adding the new iEEG dataset, and creating a combined EEG + iEEG model (see W1 of Reviewer p69F), which consumed 16x Nvidia A100 for a week. Due to the limited amount of time to respond, we were not able to evaluate the BrainBert task.

We have now revised the manuscript to include the full results on the Treebank dataset as well, including models trained on both the SWEC and the Brain Treebank dataset. Below are the results in this scenario for the Base model:

We thank the Reviewer for highlighting the strength of our cross-modality results and for acknowledging the potential impact of our findings about the advantages and disadvantages of adversarial training.

We have updated the manuscript to clarify our hypotheses about the performance of BrainCodec at low compression ratios.

---

W1. Near-perfect performance at low ratios

The generalisation capabilities of BrainCodec are indeed remarkable, as the Reviewer aptly highlighted.

We hypothesise that the vocabulary of brain states itself is consistent across patients, even if the recordings appear very heterogeneous. We now include the results obtained by training on 1, 2, and 4 patients in Appendix B.5, which show that performance does not increase markedly as the number of training patients increases. This indicates that the vocabulary is rather stable across patients and does not get stronger in representation power.

Moreover, we hypothesise that cross-patient consistency also increases as the framerate of the compression increases. When the length of each frame decreases also its heterogeneity decreases, lending itself better to being quantized. Since we decrease the compression ratio by increasing the framerate --- this empirically yields better performance rather than, e.g., decreasing the number of residuals (see Appendix B.3) ---, our results at low ratios validate this hypothesis.

Finally, as a last point of interest, we notice that BrainCodec's performance at lower compression ratios is better in set S of the BONN dataset with respect to all other sets. Following our above hypotheses, this would indicate that the brain states in set S are more consistent than other sets. Indeed set S only contains ictal activity, and the literature[1] has shown that seizures often manifest in highly regular patterns.

[1] Schindler, Kaspar, et al. "Forbidden ordinal patterns of periictal intracranial EEG indicate deterministic dynamics in human epileptic seizures." Epilepsia 52.10 (2011): 1771-1780.

---

W2. fMRI BrainCodec

We thank the Reviewer for pointing to this model with similar naming that compresses fMRI data. While this related work has been published after our submission deadline, we still would like to point out some differences, apart from the different modality:

1. The fMRI BrainCodec compresses sequences of 1024-dimensional samples with one model pass. In contrast, our (i)EEG BrainCodec compresses each channel separately, i.e., we feed sequences of 1-dimensional data into the compressor. This way, we respect the heterogeneous nature of (i)EEG, being able to compress data with an arbitrary number of channels.
2. The fMRI BrainCodec achieves a compression ration of 2.6, while our (i)EEG BrainCodec guarantees high-fidelity reconstruction at 64x construction. Such a comparison between different modalities may be difficult. However, we are confident that our methods that leverage high-quality training data and domain-specific loss functions could further improve fMRI BrainCodec, too.
3. fMRI BrainCodec could not make use of the discriminator due to the noisy training data. Our (i)EEG BrainCodec, however, leverages the discriminator to improve the reconstruction of high-frequency components.

Despite these differences, we are encouraged by the potential interpretability analysis, which fMRI BrainCodec could perform on the compressed representation. We plan to conduct similar experiments with our (i)EEG in future work, and we added a citation to the Discussion section of this submission.

Q1. EEG+iEEG model

As detailed in W1, we now provide the results of our combined EEG and iEEG model trained on both the CHB and the SWEC datasets. The full results can be found in the Results and in Appendices C and D.

---

Q2. Additional performance metrics

Following the Reviewers' suggestion, we now include four additional metrics borrowed from the signal processing domain, namely:

1. PRD of the spectrogram (PRD-spec)
2. Root-mean-square error (RMSE)
3. Signal-to-noise ratio (SNR)
4. Peak signal-to-noise ratio (PSNR)

All tables in Appendix D have been updated accordingly, showing that BrainCodec's high reconstruction fidelity translates to these four additional evaluation metrics as well.

Briefly here as an example, we report all the metrics collected for the Base model trained and tested on the iEEG SWEC dataset.

We thank the Reviewer for recognizing the strengths and potential impact of our Manuscript.

Following the Reviewer's suggestions, we have updated the Manuscript to include the results of our combined EEG+iEEG BrainCodec.

---

W1. EEG+iEEG model

We agree with the Reviewer that an EEG+iEEG model might reveal interesting insights on the behavior of BrainCodec. For this reason, we now add the results of our combined EEG+iEEG BrainCodec in the Results section and in Appendix D. We train this model using subject ID01 of the SWEC iEEG dataset and subject ID01 of the CHB EEG dataset.

First, we show the results of the mixed-modality model on the SWEC iEEG dataset.

The reconstruction fidelity of mixed-modal BrainCodec is overall inferior compared to the model trained exclusively on the SWEC iEEG dataset. This is consistent with earlier results, as iEEG-trained models always perform better on iEEG than EEG-trained models.

This is not the case when testing the mixed-modality model on the CHB EEG dataset. Here, we surpass the CHB-trained model already from a compression ratio of 4. Moreover, we always perform better than a model trained exclusively on iEEG data. This indicates that mixed-modality training is beneficial to compressing EEG data.

Overall, w e find that training BrainCodec on both EEG and iEEG improves fidelity when reconstructing EEG, without compromising performance when reconstructing iEEG. The results of the EEG+iEEG model reinforce our conclusion that transfering from high-SNR signals to low-SNR signals is beneficial.

---

W2. Mother of all benchmarks

We thank the Reviewer for indicating this benchmark. While we wish to provide as much evaluation of BrainCodec as possible, computational constraints limit us from being able to provide results on this benchmark in the given small time-frame.

Nonetheless, we believe our wide range of tasks covering both EEG and iEEG effectively characterize the behavior of BrainCodec on downstream classification.

Moreover, to further increase the size and quality of our evaluation, we now add the Brain Treebank iEEG dataset to our set of testing datasets. We find that the performance on this dataset is comparable to the SWEC iEEG dataset, pointing towards the universality of our model.

W3. Evaluation metrics

We choose three separate evaluation metrics to better characterize the performance of BrainCodec:

1. Percent-root-mean-square-difference (PRD). PRD is the most used metric to determine the reconstruction fidelity of compressors on EEG and iEEG, as seen in the literature [1,2,3].
2. Downstream classification performance. Classification tasks are a concrete benchmark for the performance of BrainCodec, as they provide an immediately parsable metric of the distortion caused by the compression.
3. Subjective expert evaluation. The specific characteristics of EEG and iEEG are not as easily parsed as music or speech. Therefore, we assess the reconstruction fidelity through the expert evaluation of a neurologist who works with this data every day and is best poised to capture the fine difference between the original and the compressed signal.

Following the Reviewers' suggestion, we now include four additional metrics borrowed from the signal processing domain, namely:

1. PRD of the spectrogram (PRD-spec)
2. Root-mean-square error (RMSE)
3. Signal-to-noise ratio (SNR)
4. Peak signal-to-noise ratio (PSNR)

All tables in Appendix D have been updated accordingly, showing that BrainCodec's high reconstruction fidelity translates to these four additional evaluation metrics as well.

Briefly here as an example, we report all the metrics collected for the Base model trained and tested on the iEEG SWEC dataset.

[1] Higgins, Garry, et al. "EEG compression using JPEG2000: How much loss is too much?." 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE, 2010.

[2] Du, XiuLi, et al. "Fast reconstruction of EEG signal compression sensing based on deep learning." Scientific Reports 14.1 (2024): 5087.

[3] Nguyen, Binh, et al. "Wavelet transform and adaptive arithmetic coding techniques for EEG lossy compression." 2017 international joint conference on neural networks (IJCNN). IEEE, 2017.

---

W4. Choice of parameters

We base our choice of parameters on extensive evaluations which we have now included in Appendices B.1 to B.4. As mentioned in W2 above, our choices of modules and parameters improve the reconstruction fidelity of BrainCodec.

---

Q1. Application scenarios

We believe BrainCodec is immediately applicable in any EEG and iEEG processing and classification pipeline. BrainCodec drastically improves data density via high-fidelity compression, paving the way for straightforward adoption in at least two scenarios.

First, for wearable, or ultimately implantable recording devices: BrainCodec reduces the number of bits that must be transferred in the air to a receiving device, with wireless communication being the most energy-consuming operation of the device. In particular, we show that even a 64x compression, BrainCodec has no negative effect on downstream tasks. Hence, BrainCodec can effectively save significant energy in communications, leading to longer battery life for long-term monitoring devices. As the encoder of BrainCodec has ~1M parameters, its implementation on such edge devices would be feasible, and its weights could be quantized for more efficient inference.

Second, in static setups, BrainCodec will lower resource requirements to store large quantities of EEG and iEEG data. Storage costs are also relevant concerns in the clinical domain. Indeed, the sharp rise in long-term recordings and healthcare-related wearable devices has also increased the rate at which electrophysiological data is collected in the clinic. BrainCodec will help mitigate these concerns in a real-time scenario as well, as it can compress and decompress up to 20 minutes of data per second on one Nvidia A100 GPU.

---

Q2. Tables

We have revised all our tables to add the new reconstruction metrics. We have also added a caption to all our tables.

By clarifying and increasing the thoroughness of our evaluation in many scenarios involved both in the methodology and the concrete application of our BrainCodec, we hope to have improved on all areas which were not satisfactory to the Reviewer.

We wish to thank the Reviewer for the constructive criticism and for recognizing the universality, the high fidelity, and the comprehensive evaluation of our proposed compression method.

We have uploaded a revised version of the Manuscript incorporating the suggested ablations and have further clarified the immediate practical applications of BrainCodec.

We briefly highlight the main contributions of this work:

1. BrainCodec is a quantized neural compressor based on EnCodec and Soundstream with specific improvements for compressing neurophysiological signal, for example the line length loss;
2. Our training methodology fully embraces the specific characteristics of scalp EEG and intracranial EEG, indicating the signficant advantages of high-SNR signals such as iEEG even when compressing low-SNR EEG;
3. Our testing methodology is exhaustive, with multiple classical fidelity metrics and expert manual evaluation by a trained neurologist. At the same time, we also evaluate on downstream classification tasks, to showcase the practicality of BrainCodec.

Moreover, during the writing of this rebuttal, we have also added the following results and contributions:

1. We have significantly improved the robustness of our ablations, including RVQ parameters, Encoder parameters, line length loss, and more;
2. We have increased the variety of the reported fidelity metrics, now including PRD, PRD of the spectrogram, RMSE, SNR, and peak SNR;
3. We have assessed the performance of BrainCodec in a mixed-modal scenario when trained on both EEG and iEEG data, and found that the performance transfer we had found from iEEG to EEG also holds in this case;
4. We have added a new iEEG dataset, the Brain Treebank datasets, bringing us to a total of 3 iEEG datasets and 3 iEEG datasets.

We now address each raised point separately.

---

W1. Methodological novelty

We base our BrainCodec on well-known and battle-tested neural compression models, namely EnCodec and SoundStream, because these RVQ-VAE architectures have demonstrated excellent results on audio data. However, audio and electrophysiological signals have little in common beyond both being time-series. Indeed, naïvely applying EnCodec to iEEG data does not yield a useful compressor.

Moreover, there are specific features of EEG and iEEG that need to be preserved after compression to maintain both signal fidelity and downstream classification performance. For example, the effects of epilepsy on the iEEG drove us to adopt the line length loss, which improves the reconstruction of both ictal and inter-ictal segments (see the next point).

We thank the Area Chair for the positive response and all the Reviewers for their constructive feedback, which helped improve our work.

Following acceptance and the suggestion of the AC, we have now revised our code and open sourced it at BrainCodec.