Self supervised learning for in vivo localization of microelectrode arrays using raw local field potential

We thank the reviewer for their positive feedback on the robustness and generalizability of our model across tasks. The reviewer raised three main concerns: model novelty, incomplete reporting, and lack of codebook interpretation. We address these concerns by clarifying the model's novelty, adding error bars to the results, and providing visual analysis of the codebook vectors.

Weaknesses:

• While prior work exists, our approach is the first to apply SSL directly on raw LFPs for anatomical decoding across species and labs.
• To address the sparse reporting, we replaced Table 1 with Supplement Figure 6, and added Figure 3 with Supplement Figure 5. These additions made sure that all accuracy values are reported with error bars.
• We agreed that deeper region results are valuable. This is our important next step to extend this method to thalamus, and whole brain regions.
• We thank the reviewer for this suggestion. We applied t-SNE to analyze the quantized vectors in the codebook and included it in the revised manuscript.

Questions:

• Following this suggestion, we expanded the broader impact section: “these methods could be applied in physiological experiments to provide real time feedback, or in chronic or clinical settings, or locating physiological events preemptively (e.g. predict which electrodes the seizure/sharp wave ripple occur), so we can do early discharges during closed loop stimulation. “
• With broader spatial coverage, we expect performance to vary by region. Potential factors affecting the performance are signal quality, number of labels (sparse labels often give poor performance), and if there exist signature biomarkers for sublayers.
• We visualized the codebook and will include them in the revised manuscript.

Limitations:

• We agree that testing in deeper brain regions (e.g.,thalamus, basal ganglia) is important for assessing generalizability. Our current study focused on cortical and hippocampus areas due to data availability. As a next step, we plan to incorporate datasets with well-labeled deep structures to evaluate whether performance and embedding consistency persist in more heterogeneous regions.

Formatting:

• We removed Table 1 and replaced it with Supplement Figure 6

We thank the reviewer for their positive feedback on the robustness of our method and its practical significance to neurophysiology. The reviewer raised three main concerns: the temporal stability of predictions and embeddings, the absence of variance reporting, and the limited explanation of design choices. We address these concerns by conducting additional experiments to analyze prediction and embedding consistency, updating our results to include error bars, and providing further clarification on our design choices.

Weaknesses:

“[Can we extend]...region classification…to a continuous measure? “

• We can extend this approach to the Allen dataset, which includes continuous anatomical labels from the Allen Brain Atlas Common Coordinate Framework (CCF), providing x, y, z coordinates for each channel. By replacing the classification objective with a regression objective (e.g., MSE loss), the model can be fine-tuned to predict spatial coordinates directly. However, the IBL and Neuronexus datasets do not provide such continuous labels, so we focused on region classification in this study.

“concerns regarding the setup and lack of variance reporting”

• We thank the reviewer for raising this point. We have removed Table 1 and replaced it with Supplementary Figure 6, which includes error bars to reflect variance in disease classification across folds of test sessions. We also added error bars to Figure 3 in the revised manuscript.

“no evaluation of embedding or prediction consistency as a function of time / state”

• We thank the reviewer for raising this point. To address this, we added additional analysis of prediction consistency over time/trials. Here the first row is the mean variance of prediction probabilities across trials, averaged across sessions, second row is the mean cosine similarity for embeddings across trials, averaged across sessions.

• The small prediction variance indicates that our model, along with other baselines, produces consistent predictions across trials. Embedding similarity, measured as the average cosine similarity between normalized embedding vectors across trials, is also high in our model, suggesting that the learned representations are stable over time.

“visualization in Fig 3a is confusing as one expects confusion matrix…overlap in text in Section 5.2”

• We agree that current Fig 3a may resemble confusion matrices and lead to misunderstanding. To clarify, we added the following text in Fig 3a caption: “Cross lab generalization matrix for zero-shot and one-shot performance, here high off-diagonal values indicate good generalization performance from one lab to another”. We also cut Section 5.2 to avoid redundancy.

Questions:

• To evaluate within session variability, we compute embedding and prediction variance across trials, as shown in Weakness # 3. Results showed that our model produces stable representation and consistent prediction with low variability across trials. Also, our model does not explicitly leverage task structure, and achieves accurate region decoding in a task agnostic manner.
• We agree that signals above the Nyquist frequency cannot be faithfully reconstructed after downsampling. But empirically, we observe that the embeddings retain some high-frequency information, suggesting that the convolutional layers do not entirely filter out high-frequency components. One possible explanation is that high-frequency bursts may leave detectable traces in the lower-frequency structure, which the model may still exploit. To further investigate this, we plan to visualize the learned convolutional filters to understand which frequency components they emphasize or suppress.
• We apologize for the oversight. This was due to changes across multiple configurations during tuning. The final model uses two notebooks, each with a size of 320, resulting in a total dimension of 640.
• We agree that linear probing would offer a clearer view of the representations' quality and will include it in future analysis.
• We thank the reviewer for this insightful point. Our model currently uses random time masking within each trial. Since inputs are 1D signals and each trial-channel pair is treated independently, no inter-trial structure is explicitly modeled.
• In response to the suggestion, we clarified this in Section 4.3: “Context tokens are the transformer outputs at masked time steps; target tokens are quantized vectors at the same time step; distractors are quantized vectors sampled from all other time steps in the same batch.”
• 5-NN majority vote is applied in a single pass. While an iterative approach could be explored, it may cause over-smoothing. In practice, one iteration is usually sufficient to correct spontaneous errors.
• To ensure recording stability, we rely on high-quality data from collaborators and preprocessing tools such as DREDGE [1] to correct for motion artifacts. As an additional consistency check, we analyzed prediction variance across time in Weakness #3 and confirmed our model's robustness. [1] DREDGE, Windolf et al. , Nature Methods 2025
• We apologize for the confusion in Figure 2d and have revised its caption as follows:: Figure 2d (left) channel-wise prediction before post processing, Figure 2d (right) example of post processing applied on lfp2vec outputs.
• To quantify the impact of post-processing, we added Supplementary Figure 7. Post-processing improves performance across all models, with the largest gain observed for LFP2vec.
• We clarified Section 5.2 to specify: “We use the same unlabeled data for SSL training and the same data with labels for supervised fine-tuning.”
• If the sampling frequency (1250 Hz) and trial length (3 s) are kept the same, the model should generalize across recording setups without reconfiguration.
• When transferring from Allen to IBL, the model is pretrained on a larger, diverse dataset. In comparison, one labeled IBL session has fewer channels and less anatomical diversity. If we fine tune on the small, biased sample, it may skew the decision boundary and bias the model away from the more generalizable Allen representations.
• To address the consistency issue, we removed Table 1 and replaced it with Supplement Figure 6 with error bars. We also added error bars for Figure 3, and the confusion matrix can be found in Supplement Figure 5.

I thank the authors for their detailed response. Their clarifications, along with the additional temporal stability analysis, made the paper stronger imo, and I therefore increase my score. Still, I would have liked to see results on the xyz coordinate prediction, instead of simply stating that it is possible given the Allen dataset, as this would have been a much more compelling result (or, to see that it can be further improved).

We thank the reviewer for their positive feedback on our novel application of self-supervised learning to source localization, and effective usage of postprocessing as a neuroscience prior . The reviewer raised three main concerns: model novelty, the value of anatomical decoding, and unclear cross-lab evaluation. We address these by clarifying differences from prior methods like BENDR, highlighting the role of anatomical decoding when MRI is unavailable, and clarifying the cross-lab evaluation setup.

Weakness:

“Other approaches have used similar wav2vec2.0 based approaches for LFP, e.g., BENDR”

• We thank the reviewer for highlighting relevant prior work such as BENDR. In the related work section, we have added the following: “Other models like BENDR [1], BrainBERT [2], EEG2REP [3], EEG-BERT [4] and related intracranial models operate on broadband signals to decode speech, behavioral, or cognitive states. While effective, these models are not designed for anatomical localization and do not incorporate anatomical priors or address cross-lab/species generalization in invasive electrophysiology. In contrast, our model operates directly on raw LFP signals and targets in vivo anatomical decoding, bridging neuroanatomy and self-supervised learning.”

[1] Kostas et al., Frontier in Human Neuroscience, 2021 [2] Wang et al., ICLR, 2023 [3] Foumani et al., KDD, 2024 [4] Wang et al., biorxiv, 2024

“Predicting a region label does not give the same information as co-registration with a pre-op MRI”

• We thank the reviewer for raising this concern. We have revised the introduction to clarify that, unlike raw LFP traces, pre-op/post-op MRI is not always available in chronic studies or during surgery. Even when available, MRI lacks the spatial resolution needed to distinguish fine subregion structures such as CA1, CA2, CA3, DG in the hippocampus.

“better performance in cross-session and one-shot …[than] the cross-lab and zero-shot. [But] in cross-session and one-shot, ground-truth labels … are already available”

• We thank the reviewer for the comment and have clarified this in the revised manuscript. All evaluations are zero-shot on new insertions or new subjects, with no ground truth labels at inference time. Cross-lab zero-shot involves no examples from the new lab and is the most challenging task. One shot uses a single labeled session but is still evaluated on unseen insertions or subjects in the new lab. What our results show is that, (1) if a lab uses similar probes and task setup as Allen/IBL, this model generalizes well in zero-shot; (2) if the task/probe is new, one labeled session is enough to enable generalization to the future subjects using the same task/probe. With more diverse lab data in the future, the next step is good zero-shot transfer to new labs.

Questions

• Yes, we revised line 191 to clarify this. Specifically, we now add: “where K(m) denotes the set of distractors unioned with the true quantized vector q_m”
• Yes, we clarified this in line 171 with the following addition: “This diversity loss is added to contrastive loss, weighted by a hyperparameter lambda that controls its strength”.
• “Temporal Smoothing” in Figure 2d refers to the method described in Section 4.4. It involves computing a weighted average of label probabilities over a time window.
• We revised the caption in Figure 2d to clarify it: Figure 2d (left) channel-wise prediction before post processing, Figure 2d (right) post processing applied on lfp2vec outputs.
• We added more labels and legends in Figure 1 Task 1 and clarified this in the caption. “Dot plot shows predicted region probabilities per trial. Columns correspond to trials, rows to channels, and colors represent brain regions (matching the legend in middle panel).
• We revised the text to define all abbreviations at first mention: IBL is the dataset collected by International Brain Lab, LFP is Local - - Field Potential signal, DC is direct current, ADC is Analog-to-Digital Converter in Neuropixel probes.
• Thank you for pointing this out. We now include in Section 4.3, “The objective closely follows the wav2vec 2.0 contrastive learning framework.”
• Apologies for the labeling error, we corrected it to “lfp2vec” in the revised Figure 3b.

We thank the reviewer for the thoughtful feedback and recognition of the practical impact and comprehensive evaluation across datasets and settings. The reviewer raised three main concerns: robustness of cross-species testing, potential clustering confounds, and limited interpretability of attention heads. We address concerns by conducting new experiments to resolve clustering confounds.

Weaknesses:

• Cross-species evaluation: We agree that a single macaque dataset focused on a motor task is limited in task diversity, and we have clarified this in the manuscript. That said, the macaque dataset presents a meaningful domain shift, across species, regions (motor vs. hippocampal/visual), and tasks (goal-reaching vs. spontaneous/visual), making it a nontrivial test of generalization. Our results serve as proof of concept, and we plan to expand datasets and analyses in future work.

• Probe-type bias in clustering: We thank the reviewer for raising probe-type and amplifier signatures as potential confounds. In cross-session evaluation, we held out entire sessions with different probes to avoid such biases. To assess their impact further, we ran two analyses: (1) linear probes predicting session ID, probe ID, or region; and (2) Silhouette scores for clustering by each factor. Both analyses showed that anatomical structure dominates the embeddings, with minimal influence from session or probe. See Question #2 for details.

• Interpretability and biological insight: We agree that analyzing embedding dimensions and attention heads would provide deeper insight. We will include these visualizations in the revision.

• Preprocessing pipeline: We thank the reviewer for pointing this out. We would like to clarify that the core preprocessing steps are described in the main text (Section 4.1, line 150-160). These steps follow the IBL LFP-band destriping pipeline, including high-pass filter, bad channel removal, rephrasing and noise reduction.

• Statistical significance: We thank the reviewer for raising this point. We have removed Table 1 and replaced it with Supplementary Figure 6, which includes error bars to reflect variance in disease classification across folds of test sessions. We also added error bars to Figure 3 in the revised manuscript.

• Novelty and broader relevance: While our method builds on wav2vec 2.0, it goes beyond simple domain transfer. Unlike BrainBERT (spectrogram-based) and EEG-BERT (focused on behavioral or physiological state decoding), we introduce a new task: anatomical localization from raw LFP signals at the channel level. To our knowledge, this is the first SSL approach for this problem. For a broader AI audience, our results show that general-purpose SSL models can effectively transfer to noisy, high-dimensional LFP recordings, an important step toward foundation models for neural time series.

Questions:

“Could you (a) add a within-species baseline…, and/or (b) report channel-wise confusion…?”

• Thank you for the suggestion. We report channel-wise confusion in Supplementary Figure 5 and will move it to the main paper. Our macaque results are within-species: trained on macaque sessions and tested on held-out macaque sessions. This shows that our audio-pretrained model can be fine-tuned for anatomical inference in a different species. True cross-species evaluation is limited by label mismatch: mouse data focus on hippocampal/visual regions, while macaque targets motor areas. Still, we are excited to explore cross-species generalization using macaque datasets with shared labels from the Allen Institute.

“Region clusters might reflect probe-type…rather than…neuroanatomy. Please test a domain-adversarial experiment … or provide a confusion table stratified by a probe model…”

• We thank the reviewer for raising this important point. To minimize the risk of probe-specific signatures driving clustering, our evaluation protocol uses a strict leave-session-out strategy. This ensures that each test session comes from a different animal and probe insertion than those seen during training, reducing the chance that the model memorizes probe-related artifacts.

• To directly test whether probes are encoded in the learned representations, we conducted two more analyses: (1) linear prober to predict region/session ID/probe ID from embeddings, (2) label embeddings by probe ID or session ID to assess whether they cluster by probes, session, or anatomical region

Linear Probing Accuracy

Silhouette Score for Clustering

• Linear probing results show that while our model embeddings encode some session and probe identity, the dominant signal is anatomical (region) information. This suggests the representations are primarily anatomy-aware, not probe- or session-biased. Also, Lfp2vec encodes significantly more region-specific information than baseline methods.

• Silhouette scores show that Lfp2vec strongly clusters by brain region, but does not cluster by probe or session, suggesting that the model learns biologically meaningful structure rather than technical artifacts. Also, Lfp2vec shows more well-separated region clusters compared to baselines.

• We will add this table and visualization of embeddings colored by probe and sessions for comparison to the supplementary. We appreciate the reviewer for the excellent ideas.

“Can you visualize Transformer attention heads…or perform a linear regression from embedding dimensions to canonical LFP bands?”

• We agree that this is a valuable approach for interpreting model internals and have made visualization of attention heads of the last layer. We will include these visualizations in the revised manuscript.

Limitations:

• More detailed Alzheimer's results are shown in Supplement Figure 6. We acknowledge that the demo is preliminary in mice and not validated for clinical or diagnostic use in humans. To deploy this clinically, we would need large scale, multiple site studies, cross-species validation, comparison with clinical biomarkers, FDA approval, and human trials with reproducibility studies. We appreciate the reviewer’s concern, and will revise the broader impacts to reflect careful consideration of clinical implication of the model performance.
• Cross-species results are shown in Supplement Figure 5. We acknowledge that this is only within species study on maceque, and performance on human subjects remains unverified and may require further changes.
• We appreciate the reviewer for bringing up the concern regarding privacy. Though our research does not involve human recording experiments now, we recognize that privacy becomes a critical issue when scaling such approaches to human data. For future research, we could remove all direct identifiers, mask metadata, and only release pretrained models or de-identified features instead of raw LFP traces.

Formatting: SiNAPs probe stands for Simultaneous Neural Active Pixel Sensor probe