Thank you for taking the time to review our work! We appreciate the comments, which have helped us improve the paper. We address your questions below:

1. how exactly the position of each channel is defined in spatial position encoding…The notations used in position embedding are not defined.

   • Thanks for the question! We have clarified the text in response to your feedback. Each channel has a 3D coordinate, as given by the datasets from BrainBERT and TUEG respectively. For the iEEG dataset, these coordinates are locations in the brain volume. For the EEG dataset, they are locations on the scalp. We can represent each of these three coordinates as vectors, using the BERT position encoding scheme. We concatenate the three vectors together to get a representation of the spatial position. We have added clarification on the notation to the text.

2. The experiments are somehow limited…more tasks, e.g., TUEV in the TUH EEG corpus, should be conducted.

   • Since our goal is to study multi-channel aggregation, we evaluate using intracranial and EEG data that contain multi-channel data. The EEG dataset mentioned (TUEV) only contains single channel data. It’s worth noting that most other approaches in this space don’t bother to evaluate using multiple datasets, let alone multiple modalities $1,2,3,4$ . We evaluate using four audio-linguistic tasks from an intracranial dataset and one abnormal EEG detection task, which together cover a wide range of signal morphologies.

3. …not straightforward to understand how the connectivity is measured…More detailed explanations and rationales should be provided

   • Thanks for your feedback! We’ve revised the description of connectivity (Appendix G), added a diagram (Figure 12), and added a pseudocode algorithm (Algorithm 1). In short, the intuition is this: during pretraining, the model learns a channel-wise task that requires it to predict channel information by relying on the context of surrounding channels. Then, after pretraining has finished, we can use the model to query which pairs of channels are important to each other. We do this by selecting a channel, omitting it from the model input, and then measuring which of the model predictions on the surrounding channels are most affected.
   • To provide further context/comparison for these connectivity results, we have added the electrode connectivity matrices, which can be seen in Supplementary Figures 13 and 15. These are given both for our method and a traditional method (coherence/cross-correlation). We have also computed the correlation between these matrices, and these can now be seen in Supplementary table 7. The average correlation is 0.51.

4. Page 3, line 161: According to the paper's description, the input to the temporal encoding is a single value of a channel at a time t. Is this correct?

   • $x^t\_i$ does not represent a single value in time, but rather an interval of time that begins at $t$. It is actually a vector in $\\mathbb{R}^{T}$ where $T$ is the number of time samples in the interval. We now clarify this in the text.

References

$1$ Zhang, Daoze, et al. "Brant: Foundation model for intracranial neural signal." Advances in Neural Information Processing Systems 36 (2024).

$2$ Jiang, Weibang, Liming Zhao, and Bao-liang Lu. "Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI." The Twelfth International Conference on Learning Representations.

$3$ Ye, Joel, and Chethan Pandarinath. "Representation learning for neural population activity with Neural Data Transformers." Neurons, Behavior, Data analysis, and Theory (2021).

$4$ Wang, Christopher, et al. "BrainBERT: Self-supervised representation learning for intracranial recordings." The Eleventh International Conference on Learning Representations.

Additional questions

• The paper claims to learn representations of neural activity; do the authors mean the final output of the PopT or only the CLS output? Can one also learn a neural representation without using a CLS token?

  • Both the token outputs and the CLS output are meant to be included when we discuss the resulting representations. The CLS token is used for our decoding experiments and the token outputs are used in the connectivity analysis.

• Can this representation be thought of and used as a dimensionality reduction method?

  • Yes! Good question! The original BrainBERT paper explores the intrinsic dimension of the learned single channel representations, and this is what we would like to do with our multi-channel representations in the future.

• Who are the intended users of this method?

  • Neuroscientists and brain machine interface (BMI) researchers: for use in decoding and interpreting brain data.

• Can this method be used for non-human invasive research, such as calcium imaging, neuropixels, etc?

  • For the modalities proposed, because all the activity is recorded from neurons that are physically close together, it’s unclear if the 3D position will be meaningful information. This matters because we find that 3D position is critical information for iEEG and EEG decoding, so it is an open question as to whether this approach would translate effectively.

• Is it possible that the discriminative nature of the pre-training objective leads to potentially misleading representations in case of noisy/faulty channels?

  • There’s merit to this concern. Whereas a generative model would (likely) not learn to produce outputs that look like noise, for a discriminative approach, there’s a chance that out-of-distribution faulty channels could land in the same latent space region as non-faulty channels. However, this becomes less likely as the amount of pretraining data grows, and more kinds of noise channels become attested to in the data. Generally, noisy channels also provide less information when performing our SSL tasks, so we expect that our model to rely less on temporal embeddings that look like noise.

Minor comments

• line 163: sinusoidal position encoding: is there a simple formula or a couple of words to explain this method without necessarily having to read the cited paper?

  • Yes! We include a brief description
• Thanks for the other catches!

References

$1$ Vaswani, A. "Attention is all you need." Advances in Neural Information Processing Systems (2017).

$2$ Devlin, Jacob. "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805 (2018).

$3$ Wang, Christopher, et al. "Brain Treebank: Large-scale intracranial recordings from naturalistic language stimuli." NeurIPS (2024).

Interpretability

• It is not immediately clear how one infers (pairwise) connectivity by masking out the activity of one channel at the time and measuring the degradation in channel-wise loss. Can you add a few details on that?

  • Thanks for your feedback! Given your feedback, we’ve revised the description of connectivity (Appendix G), added a diagram (Figure 12), and added a pseudocode algorithm (Algorithm 1). In short, the intuition is this: during pretraining, the model learns a channel-wise task that requires it to predict channel information by relying on the context of surrounding channels. Then, after pretraining has finished, we can use the model to query which pairs of channels are important to each other. We do this by selecting a channel, omitting it from the model input, and then measuring which of the model predictions on the surrounding channels are most affected.

• And do we learn anything new from this analysis as compared to previous coherence or cross-correlations analyses? Can we have a few more qualitative and/or quantitative comparisons?

  • To provide further context/comparison for these connectivity results, we have added the electrode connectivity matrices, which can be seen in Supplementary Figures 13 and 15. These are given both for our method and a traditional method (coherence/cross-correlation). We have also computed the correlation between these matrices, and these can now be seen in Supplementary table 7. The average correlation is 0.51. It seems that the methods arrive at overlapping pictures of connectivity, especially along the strongest points. But in general the PopT seems to discover sparser connectivity maps.

• Same questions go for the functional brain regions from attention weights. Moreover, what are the possible caveats of inferring connectivity or functional regions from PopT?

  • The authors of the original dataset released analyses where they identify word responsive electrodes (see fig 2h-i in $3$ ) using t-tests between pre- and post- word-onset activities. We find a correspondence between regions with a high fraction of word responsive electrodes and the regions with high attention weight, as found by our analyses. We find a Pearson correlation coefficient of 0.4 against the fraction of significant electrodes found in the prior work. It would be good future work to investigate the additional electrodes found to be highly attended to in a trained PopT to evaluate what features may be additionally leveraged for improved decoding.

Thank you for taking the time to make a close reading of our work! We appreciate the helpful suggestions. We’ve made revisions accordingly:

Clarity

• in many points … it is not clear what the dimensions are of the variables…what is the dimension of $X^t\_B$? Is the “+” used for concatenation or for an actual sum? What is the output dimensionality of the temporal embedding model B?

  • The temporal embedding model $B$ outputs vectors of dimension $d$. The input to the PopT is a collection of such vectors that have been summed (not concatenated) with their position information. Then, the input $X^t\_B$ can be written as an $S \\times d$ matrix where $S$ is the number of channels. We have revised the text to clarify this and the dimension of other variables.

$CLS$ token: it is stated nowhere that this is the token used for downstream decoding/classification tasks, and what space it belongs to. Is it only binary? Please clarify in the text or figure legend.

• Thanks for pointing this out. We clarify this now in section 3 of the text under “Fine-tuning”.

• Self-supervised Loss (line 173-197): the losses are described in words, but it would be useful and much clearer to write them in formulas. Right now, this paragraph is a bit messy and somewhat colloquial.

  • This is a helpful suggestion! We now give a complete formal description of the losses in Appendix A.

• Fine-tuning (line 199-201): colloquial and relies on details only available in the appendix. It would also be useful to formally define what is meant by a decoding task.

  • We have re-written this section!

• Figures: Fig 1 is unclear and messy. Why does it go from bottom to top, rather than the opposite?

  • Thanks for the comments. We’re working on cleaning up Figure 1. We follow the convention of showing information flow from bottom to top (cf. the original Transformers $1$ and BERT $2$ papers).

• Currently, the “Population Transformer inputs” title of Fig 1a is just above the output, making it very confusing. Moreover, the figure has at least four different font sizes.

  • We updated figure 1 for clarity and to standardize font sizes.

• The “+” used in temporal embedding + 3D position is confusing; does it refer to concatenation or addition?

  • Addition. We now clarify this in the figure caption.

• The title of the colorbar near the STFT is nearly illegible.

  • Removed colorbar, since we are mainly interested in presenting a schematic

• Fig 3-4: font sizes widely different across figures; Fig 3 text is hard to read on print. Same for Figs 6-7-8.

  • We’ve increased figure size and font sizes!
  • Figure 8: We abbreviate the region names, so we can display them at a larger size and add a very visible legend

• Fig 8 left title is partially covered by the plot.

  • Fixed! Thanks for the catch!

Thank you for your review of our work! We appreciate the feedback and address the comments below:

1. Despite the extensive experimentation, the paper lacks a clear introduction to the field and a focused statement of its goals. Important background information and foundational definitions are often buried in the appendix rather than integrated into the main text.

   • Thanks for the feedback! We’ve revised our introduction to start with a clear statement of our motivation and goals, with care to explain the relevant prerequisite definitions.

2. For instance, a key innovation of the study is the use of channel-level and ensemble objectives. Providing a more detailed literature review on current approaches to these objectives would help the reader understand the limitations of existing methods and the advantages of the proposed improvements.

   • We will emphasize the field’s exploration into channel-level aggregation in the related works and background. To our knowledge, most deep learning approaches take an end-to-end training approach as outlined in our Related Works, and only a few have explored channel-level aggregation. Those that do focus on supervised training, rather than tackling the problem of leveraging unannotated data from variable subject layouts.

3. The connection between experiments is also unclear…a clear description of the data used for pretraining.

   • Given your feedback, we’ve revised and done our best to fit more information about the data and PopT into the main text.

4. L1 reconstruction loss…is not discussed in the text…appears tangential to the core contributions… $suggest$ omitting

   • Thanks! We now explicitly discuss this in the text. The experiment shows the necessity of a discriminative loss, rather than a reconstructive loss, as is used in other self supervised approaches, e.g. BrainBERT. Since we consider this an important point, and given that it is only one line, we would like to keep this result in the main text! But given your feedback, we will remove the closely-related experiment (L1 + discriminative loss) since it is mostly redundant.

5. While Chronos/TS2Vec and BrainBERT/TOTEM are used as benchmarks for PopT, it remains unclear why these specific models were selected. The authors could strengthen the study by discussing the criteria for choosing these benchmarks.

   • We have revised to add a brief description/reason for including each encoding model. In short, they cover a wide range of encoding motifs: convolutional (TS2Vec), tokenizing (TOTEM), transformer based (Chronos), and iEEG specific (BrainBERT).

6. In Figure 8, the plot slightly overlaps with the title on the left image

   • Thanks for the catch! Fixed.

7. Table 3 refers to “PoPT w/o group-wise loss”, is the group-wise loss the same of ensemble-wise loss. To improve clarity, could the authors consider using consistent terminology throughout?

   • Thanks for the catch! Yes. They are meant to refer to the same thing. We have updated the text to be consistent.

8. It would be insightful to see how one of the baseline models, specifically BrainBERT from Table 1, performs on the hold-out dataset. Could the authors report the performance of BrainBERT on the hold-out dataset in a format similar to Figure 6?

   • This is a very helpful suggestion! The BrainBERT performances on the test dataset have been added to Figure 6. We also run the same hold-one-out analysis on a PopT trained with TOTEM, which we show in Appendix E.

Thank you for taking the time to review our work! We appreciate your feedback and questions, which we address below:

1. In Tables 1, 2, and 3, consider adding statistical testing

   • Done! In tables 1 and 2, we use a Wilcoxon rank-sum test to compare between the first place and second place models in each section. For table 3, we use Dunnett’s test to pick out which ablations are significantly impactful.

2. Could you explain how you construct pairs?

   • Pairs consist of the activity from two different subsets of channels, each subset coming from a specific time point. The time points can either be consecutive times (positive pair) or randomly selected from any other time point (negative pair). Here, “consecutive” means “occurring 500ms afterwards”. In the case of positive pairs, we can ensure that the activities are consecutive by construction: we simply take activities from two windows, separated by 500ms. We clarify this in the text.

3. short-time Fourier transform…did not find it to be discussed anywhere in the text.

   • The STFT is part of BrainBERT’s preprocessing. We have revised the text to clarify this and explain how it fits into the broader pipeline.

4. …how much $does$ Gaussian fuzzing contribute to the final performance?

   • Good question! We’ve added an ablation experiment (see updated Tables 3 and 8). We find that Gaussian fuzzing provides small, but consistent benefits across decoding tasks.

Thank you for your response! We appreciate the interest in the problem space and your question regarding connectivity metrics and analysis:

• …how does the connectivity shown compare with cross-correlation…?

  • For comparison, we include traditional coherence analysis, which is simply cross-correlation, taken across different frequency bands and then averaged. This is shown on the left hand side of Figure 8. We have added more typical channel-connectivity matrices in Supplementary Figures 13 and 15. We show these both for our method and for cross-correlation (coherence). For each pair of matrices, we measure the correlation between our method and the traditional method. These can be seen in our newly added Table 7. The average Pearson correlation coefficient is 0.51.

Minor Formatting

26. 214-215: Could you please remove the parenthesis inside the parenthesis?

    • Done!

27. Figure 3 is referred to before Figure 2 in the text. Could you please renumber the figures to maintain clarity?

    • Done! We reordered the sentences in the text so that the figures are referred to in order.

28. Table 1: Could you please emphasize your result better by bolding the label Pretrained PopT or star it for "best overall"?

    • Done! See updated text.

29. Results 264-269: hard to read. Could you please remove the parenthesis inside the parenthesis?

    • Done! See updated text.

30. Table 1 and Figure 1: Could you please make latex keep them inside the section where they are referred to?

    • Done! Table 1 and Figure 1 are now in their respective sections where they are first mentioned.

References

$1$ Wang, Christopher, et al. "Brain Treebank: Large-scale intracranial recordings from naturalistic language stimuli." NeurIPS (2024).

$2$ Bommasani, Rishi, et al. "On the opportunities and risks of foundation models." arXiv preprint arXiv:2108.07258 (2021).

$3$ Berezutskaya, Julia, et al. "Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film." Scientific Data 9.1 (2022): 91.

Clarifications

13. Abstract 16-18: The sentence "lowers the amount of data [required] for downstream decoding experiments" is compatible with the presented results (Fig. 4 and 5). However, the verb 'lowers' requires a comparison group, and 'downstream decoding experiments' is vague. The meaning of 'downstream' is also ambiguous. Could you please rephrase this?

    • Here, the implicit comparison group is "the amount of data that would otherwise be needed" in the models we compare against.
    • "Downstream" and "downstream tasks" are standard terms when discussing foundation models (cf. $2$ )

14. Calling the PopT computationally lightweight in comparison to end-to-end approaches is misleading because the PopT only works with pretrained weights from BrainBert.

    • PopT can be trained on top of any pretrained temporal embedding, not just BrainBERT. In our experiments, we pretrained with TOTEM, TS2Vec, and Chronos.

15. Abstract 18-20: Calling the PopT "more interpretable" when not comparing the interpretations to existing traditional neuroscientific methods for interpretability is misleading. Could you please rephrase this?

    • This is fair. We mean "interpretable" as demonstrated by our study of the attention weights and connectivity. For strict correctness, we will remove the word "more" since we did not make an explicit comparison with other methods.

16. Abstract 18-20: What does 'retaining competitive performance' mean? In all iEEG data, it outperforms the end-to-end method used, in all EEG data it is outperformed by LaBraM. Could you please provide this information without ambiguity?

    • Here we simply mean that PopT's performance is always at least in the ballpark of recent end-to-end methods. We have revised our abstract to say "similar or better performance".

17. What are "massive" amounts of data?

    • The dataset we use contains 4,551 electrode-hours of recordings (see original BrainBERT paper). For context, it's rare to have this amount of intracranial data for the movie watching setting. The most similar dataset to the dataset we use, Berezutskaya et al. $3$ , presents participants with vastly less stimuli: a 6.5 minute short movie, compared to the average of 4.3 hours of movie per subject in the data we use.

18. Discussion 524-525: " $…$ could provide an even higher resolution latent space $…$ ". Higher than what? Could you please rephrase this?

    • Sorry; that was confusing as previously written. We've revised that section to clarify.

19. The codebase is messy and lacks clear examples, instructions, or tutorials. Is this a work in progress or the cleaned-up codebase?

    • We respond to this point above and in point 12. In short, we have cleaned up the codebase and it can be seen in the updated supplementary materials. As we are actively developing the code, it remains a work in progress.The codebase, with documentation and tutorials, will be complete by the time of the conference. As it stands, we have made the code base neater, and outlined the minimal set of commands needed to run PopT.

20. The explanation is not reader-friendly and not comprehensible for someone who does not know the types of neurorecordings.

    • Thanks for the feedback! We've revised that section to explain the differences between the types of neurorecordings in more detail.

21. these are rather "contrastive" components and not "discriminative" components.

    • We use the descriptor "discriminative" to reflect the fact our objective function formula is given in terms of classification, i.e., a single prediction is made for each single input token. In comparison, the objective function formula for a contrastive approach like SimCLR takes positive and negative examples as inputs and explicitly contrasts them using an interaction term.

22. Figure 2, 6, 7: Could one use violin (or box) plots instead of bar plots to convey the variability and scatter the individual data points on top...?

    • We have created updated Figures 6 and 7! We keep Figure 2 as a bar plot for visual clarity, since it does not fit on the page as a violin/box plot.

23. "potentially due to adaption to the temporal embeddings used" meaning unclear. Could you please rephrase this?

    • We have re-written that entire section following the feedback on points 3,4,5.

24. These sentences seem contradictory, which might be easily resolved with rephrasing them. Could you please rephrase this? Part of the Figure 5 caption could go here into the main text.

    • Yes, that was confusing as it was written. We have revised that section.

25. What is 'highly' sample efficient?

    • We just mean that the PopT can achieve the full performance of the baseline models after having seen $\\approx 10$ of the dataset.

Thank you for taking the time and effort in reviewing our work! We truly appreciate the feedback and opportunity to further improve the paper.

1. calling PopT computationally lightweight in comparison to end-to-end trained models is misleading since it relies on pre-trained, e.g., BrainBert weights, which already bear significant computational demands.

   • In fact, this is exactly what we mean by “computationally lightweight”! Our main argument is that the research community has already produced many pre-trained weights for single-channel temporal embeddings, and this means there is an opportunity to cheaply learn multi-channel embedding by aggregating across single-channel representations. Our presented approach is lightweight compared to end-to-end models which require backpropagation through the entire temporal encoding and spatial encoding stack. To make this point precise, we show a comparison table of trainable parameters between PopT and other end-to-end models in Appendix B.

2. "more interpretable," … lacks precision without clear comparison groups or quantitative metrics… the interpretability analysis does not include any comparison method…

   • By “more interpretable” we just mean to say that end-to-end models lack an attention weight matrix across individual channel units. But for the sake of strict correctness, we will drop the word “more” since we do not make an explicit comparison with end-to-end model interpretability methods.

3. In addition, the generalizability claims lack thorough validation: the hold-one-out subject analysis does not sufficiently establish robustness. This experiment could benefit from a broader validation, e.g., cross-validation approaches.

   • We apologize for the confusion that resulted from our sparse text description. The hold-one-out results are, in fact, cross-validated. That is, we do not draw a single subject to test on, but instead, we hold out all test subjects in turn from pretraining and then evaluate on the held out subject. The error bars reflect the variation across held-out subjects. We now clarify this in the text.

4. the codebase requires significant improvements, as it is messy, poorly documented, and lacks proper instructions or tutorials.

   • We have cleaned up the codebase and it can be seen in the updated supplementary materials. Since we are actively developing the code, it does remain a work in progress. The codebase, with documentation and tutorials, will be complete and ready for public release by the time of the conference. As it stands, we have made the code base neater, and outlined the minimal set of commands needed to run PopT.

Responses to Major Questions:

1. You show that the decoding performance increases with more subjects and decreases with holding out a subject. However, these experiments are limited because they consider a validation on specific channels of one specific subject, or holding out one specific subject. Is this also valid across arbitrary channels and arbitrary subjects?

   • However, these experiments are limited because they consider…one specific subject.

     • We respond to this more fully in addressing your points 3,4, and 5 (see below). But in short, the hold-one-out analysis from Figure 6 actually does show cross-validated results, not the results for one specific subject. We have made this clearer in the text now.

   • Is this also valid across arbitrary channels …?

     • PopT takes multiple channels as input, and all subjects are each eventually evaluated on the full 90 channels of input (see the rightmost end of each line in Figure 3). One could still be concerned that we have gotten lucky with our specific ordering of electrode subsets.To this end, we have added a plot in Appendix D which shows results for randomly selected electrode subsets.

2. Could you report the number of free parameters and nonlinearities of all networks and factor this into the discussion of your results?

   • We provide a comparison of the free parameters with our baseline models and SOTA deep learning models in Appendix B Table 4. The DeepNN baseline and our PopT contain a similar number of trainable parameters. Both the DeepNN and the PopT use non-linear GeLU activation functions.

3. What if the held-out subject was a lucky draw? Can this comparison be done in a crossvalidation?...Can the performance decrease be computed for hold-k-out and presented as a function of k?...I would have expected that "All subjects" and "0 subjects" correspond to "All" and "Non-pre-trained PopT" but the numbers are different…Figure 7: What if "across channel ensembles 5-30 on a held out test subject" is a lucky draw?

   • We address your points 3, 4, and 5 as a unit. These are valid concerns. And we see that our sparse descriptions have caused understandable confusion. First, to clarify the status quo:
     • The hold-one-out results in Figure 6 are, in fact, cross-validated. That is, we do not draw a single subject to test on, but instead, we hold out all test subjects in turn from pretraining and then evaluate on the held out subject. The error bars reflect the variation across test subjects. We now clarify this in the text.
     • The scaling results in the (old) paper’s Figure 7 were not cross-validated. We draw a single subject to test on, and progressively add more subjects to the training. All evaluation is done on this single subject. This is the reason why the numbers for “0 subjects” and “Non-pretrained PopT” do not match between Figures 6 and 7. We used this design, because it was not feasible to cross-validate when holding out more subjects, since each pretraining run takes about half a week, and the number of pre-training runs that are required for an exhaustive hold-k-out analysis grows combinatorially as a function of k.
   • In the case of the scaling results, your concern about a lucky draw is fair.
     • To this end, we have run a new experiment in which we create truncated datasets to pretrain on. You can see the updated figure in the revised paper’s Figure 7. Now, since the independent variable is the percentage of pretraining data, rather than the number of subjects, we do not encounter the combinatorial problem described above. The performance is now evaluated on all test subjects, the same as in the hold-one-out results.
4. See 3.
5. See 3.

6. To say $more informative$ , I would expect an analysis of representations resulting from different loss functions.

   • By “informative” we only mean “more beneficial for decoding,” as evidenced by the result of the ablation experiments. We have revised the text to clarify this.

7. Can one present the raw functional connectivity matrices inferred this way in addition to the aggregate?

   • Done! The raw channel connectivity matrices can now be seen in Appendix H: Connectivity.

8. Can one compare the results to the functional connectivity inferred with traditional cross-correlation methods on the measurements

   • The correlation between the connectivity matrices obtained via PopT and the matrices obtained via coherence analysis can now be seen in Appendix H: Connectivity. The average Pearson’s r correlation across test subjects is 0.51.

9. Can one compare to neuroscience ways to characterize the activated brain area?

   • The authors of the original dataset released analyses where they identify word responsive electrodes (see fig 2h-i in $1$ ) using t-tests between pre- and post- word-onset activities. We find a correspondence between regions with a high fraction of word responsive electrodes and the regions with high attention weight, as found by our analyses. We find a Pearson correlation coefficient of 0.4 against the fraction of significant electrodes found in the prior work.

10. What are differences in, e.g., architecture or training that makes LaBraM outperform the PopT? Could one provide possible explanations for why LaBraM outperforms the PopT on EEG decoding (whereas end-to-end approaches specifically designed for iEEG do not)

    • LaBraM is designed specifically for EEG, so it is not surprising that it performs well in this domain. LaBraM leverages EEG specific information such as power spectral densities as part of the temporal encoding, and was pretrained on a wide variety of EEG data. It is rather surprising that PopT remains competitive with LaBraM, despite not being end-to-end, nor specialized for EEG (we only pretrain on a single EEG dataset and use general off-the-shelf time-series encoders).
    • Although Brant was specifically designed for iEEG, it was trained on lower sampling rate data and only leverages MLP aggregation of electrode information. These design choices may have hindered its ability to perform on our more challenging decoding tasks.

11. there is a much more significant improvement for the sentence onset and speech vs. non-speech tasks over the improvement in the pitch and volume tasks. Could one provide an explanation for this in the results section?

    • Regions of cortex are functionally specialized, so the sampling of electrodes will determine the decodability for each task. The electrodes in the intracranial dataset are sampled heavily from speech processing areas, namely the superior temporal gyrus. This performance trend can also be seen in the original BrainBERT paper’s findings for the frozen BrainBERT embeddings, for which the Sentence onset and Speech vs. Non-Speech had a mean ROC-AUC of 0.66 and 63, while the Pitch and Volume tasks had 0.51 and 0.60. We revised the text to mention this.
12. Codebase is cleaned and updated (see new zip file and author response part 1 for more elaboration).

The authors have addressed my concerns and questions rigorously and convincingly. I have increased my evaluation to 8: accept, good paper. Thank you for your hard work.