Learning to cluster neuronal function

Thank you for appreciating our paper, noticing novelty and validation “on a wide range of datasets, across species and visual areas” and toy data.

We are happy to address you comments and questions below:

Questions

Q1: Additional experiment on marmoset retina: Thanks a lot for suggesting an additional experiment with varying the number of clusters on the retinal ganglion cell dataset. The table below reports ARI values per cluster (and std across 3 seeds). We indeed observe a clear peak around 4 clusters, as it should be for this retina (Fig.3 in the original manuscript), so ARI does “peak noticeably at the ‘true’ number of clusters if such a structure exists.” We will add this experiment as a figure into the final version of the paper.

Q2: Multiple sessions: Actually, our method is capable of handling data from different subjects/trials altogether, in fact, it is already done for mouse visual cortex and macaque V4. We do indeed handle sessions separately for mouse and marmoset retinas due to the differences in the experimental conditions (presumably different temperature) which influences the speed of the responses so much that this would be the most trivial dimension to separate the neurons for different groups [1,2]. While several works try to address this issue and integrate several retinas into the same model [3,4,5], this is still an active area of research, and is not the main focus of our paper.

Weaknesses

W1 Hyperparameters: Our method indeed introduces additional hyperparameters, but not all of them need extensive tuning. Specifically, β just scales the clustering loss to the same order as the Poisson loss, which can be done with 1-2 short runs. However, finding the optimal pretraining epoch or number of clusters indeed increases the amount of necessary experiments. But since there are no competing methods which achieve the same without hyperparameter tuning, we believe that this degree of tuning is manageable and not unusual.

W2 Multiple sessions: See Q2

We thank you once again for constructive feedback and helping us to improve our paper. We are happy to answer any follow-up questions.

References:
[1] Zhao, Z., Klindt, D. A., Maia Chagas, A., Szatko, K. P., Rogerson, L., Protti, D. A., ... & Euler, T. (2020). The temporal structure of the inner retina at a single glance. Scientific reports, 10(1), 4399.
[2] Fritsches, K. A., Brill, R. W., & Warrant, E. J. (2005). Warm eyes provide superior vision in swordfishes. Current Biology, 15(1), 55-58.
[3] Höfling, L., Szatko, K. P., Behrens, C., Deng, Y., Qiu, Y., Klindt, D. A., ... & Euler, T. (2024). A chromatic feature detector in the retina signals visual context changes. Elife, 13, e86860.
[4] Vystrčilová, M., Sridhar, S., Burg, M. F., Gollisch, T., & Ecker, A. S. (2024). Convolutional neural network models of the primate retina reveal adaptation to natural stimulus statistics. bioRxiv, 2024-03.
[5] Gonschorek, D., Höfling, L., Szatko, K. P., Franke, K., Schubert, T., Dunn, B., ... & Euler, T. (2021). Removing inter-experimental variability from functional data in systems neuroscience. Advances in Neural Information Processing Systems, 34, 3706-3719

Thank you for your thoughtful feedback, highlighting that we are solving an important problem with a "clear, hypothesis-driven framework". Below we address your specific comments from the review:

(1) Limited Novelty:
Of course our method builds on prior work (DEC, EM, TMM), but it introduces key innovations: The novelty is the combination of solving a multivariate non-linear regression task (how neurons respond to images) jointly with clustering the neurons’ readout weights. Unlike DEC, which clusters input samples by altering the model’s latent space and enforcing unit-scale clusters, we cluster model weights directly, constrained by the Poisson loss, which is jointly optimized with the clustering loss. This requires significant adaptation, including introducing EM updates, learning the scale matrix and an appropriate curriculum of first pre-training for the regression task. DEC serves more as inspiration than as a direct combination of methods. Our model provides a hypothesis test for a core neuroscience question—whether distinct functional cell types exist in mouse V1. Currently no such method exists! The components may be known, but their integration, adaptation, and application are novel and scientifically meaningful.

(2) Method Complexity:
DECEMber has several hyperparameters, but not all need tuning. For example, β just needs to roughly balance clustering and Poisson losses, which can be estimated in 1–2 few epochs runs. In several experiments (marmoset, mouse retina, monkey V4), we did exactly this, without sweeping β. Regarding Figure 4: It is correct that very large β>=10^5 hurt performance (Fig. 4F). However, consistency improves substantially compared to the baseline already with much smaller β~=10^4 (Fig. 4H), in which case the high predictive performance is maintained. Thus, there is no trade-off. We will improve the figures to make this clearer in the final version. The experiment of increasing the clustering weight, β, to the point where performance drops is intentional. It demonstrates that aggressively forcing a clustered structure can erase functionally relevant details, but does not improve embeddings’ consistency. This result supports our conclusion that excitatory neurons in the mouse visual cortex likely form a continuum rather than discrete types. It is not a weakness of the method – it is a property of the data! Finally, t-SNE plots were used only for visualization, not for tuning β—we’ll make this clearer in the text.

(3) Tedious Tuning:
You're right that careful hyperparameter selection matters and we appreciate the opportunity to clarify DECEMber’s practicality and robustness. We believe that the degree of tuning required is manageable. Even sub-optimal hyperparameters yield substantial benefits. The ARI for mouse V1 data does peak at 10 pretraining epochs, but Fig. 5C shows that nearly all tested settings outperform the baseline across a range of cluster counts. This suggests DECEMber offers more consistent embeddings than standard methods, even without a perfectly tuned grid search. On other datasets (mouse retina and macaque V4), we explicitly avoided extensive tuning—only making minor learning rate adjustments for stability and choosing β as described above (1 additional run to estimate clustering loss order) and 3 different pretraining epochs to try out—and still observed a doubling of ARI without compromising predictive accuracy (see App. A6 for predictive performances). This highlights DECEMber's robustness and strong out-of-the-box performance. We acknowledged these limitations in our discussion. As with many flexible deep learning methods, some tuning is expected, but the resulting gains in embedding stability and interpretability justify the effort.

(4) Value of the Method:
You ask “If the core problem the method aims to solve does not have a clustered solution, what is the point of introducing this complex clustering mechanism?” – The point is that it is an open scientific question whether the solution is clustered. Neurons are widely thought to form discrete types morphologically and genetically, and in the retina, this holds for function as well [6]. But whether V1 neurons cluster functionally is unknown. Our method tests this: if discrete types exist, clustering should improve model structure or performance (lines 314–316). This holds in the retina (Fig. 3), where functional types are known. Moreover, as reviewer gTJC suggested, varying cluster count for marmoset retina reveals a clear peak at the ground-truth: 4 clusters, again showing DECEMber capability to reveal distinct cell types if they exist.

That is, the marmoset retina dataset provides the positive control that our method does what it is supposed to do. In mouse V1 (Fig. 4), in contrast, clustering does not improve predictive performance and ARI stays below 50%, with no clear peak — suggesting a functional continuum. This negative result is meaningful and supports recent findings [1-3]. Rather than undermining the method, it highlights the method’s value for quantitatively testing, not assuming, biological structure.

(5) Methodological Issue:
Clipping small values is a common and practical approach to address numerical stability and it worked well in all our experiments. This is really a small detail that has no practical implication for the method.

(6) Baseline Comparison:
We appreciate your careful observation. You note that “comparing an end-to-end trained method like DECEMber with a post-hoc GMM clustering on unstable embeddings is unfair” — This is precisely the motivation for our method, because no other end-to-end method for this problem exists. Therefore, we believe that comparison with post-hoc baselines is both necessary and meaningful. If you are aware of alternative baselines that would offer a fairer comparison, we would be grateful for suggestions.
To be clear: Across multiple DECEMber runs, the random seed of the predictive model's parameters also varies. The fact that combining training the predictive model and clustering leads to more consistent embeddings (and, hence, clustering) is one of our main contributions.

(7) Visualizations:
We will improve the figures and captions.
1 -We will add the suggested plot to the manuscript. As we cannot add plots for the rebuttal, below is a table. It shows that for a fixed number of clusters when β is increasing ARI increases as well and the predictive performance stays roughly constance before β>=$10^5$, where we see a rapid drop in performance without any gain in ARI.

2-4 - Fig 7 caption will be : “ARI across three seeds per model. A: Mouse retina [19], averaged over six models reflecting different experimental conditions (see App. A.7). B: Monkey V4 [51], three seeds. DECEMber improves ARI across both datasets and architectures, consistent with mouse V1 (Fig. 4H), supporting its generalization.”

We will extend Fig. 7A/B to 60 clusters for consistency with Fig. 4H. However, the evaluation method is the same across Figures 4H, 7A, and 7B - all ARI values are computed over three seeds per model. For the mouse retina, we train six separate models to account for different experimental conditions (temperature), which strongly affect response speed and would trivially separate neuron groups [3,4]. ARI curves are averaged over three seeds and weighted by neuron count. See Appendix A.7 for details and per-model ARIs.

Overall, these results further support our finding: DECEMber improves consistency across diverse datasets and models, when enough neurons are available.

(8) Ambiguous Contribution:
We appreciate your concern as it's an important point to understand. First, to clarify: in Fig. 4F, the best ARI on mouse V1 is ~0.40 (not 0.25) for 5 clusters, with a baseline of ~0.25 (not 0.15) — a ~60% improvement, consistent across cluster counts.

That said, we agree the absolute ARIs are low, as are other metrics (e.g., Fowlkes–Mallows score ~0.5, Appendix A.4), even when the clustering bias is strong. But this is not a model weakness — it’s the core scientific finding: no method, including DECEMber, reveals clear, stable clusters. This supports the idea that mouse V1 excitatory neurons form a functional continuum rather than discrete types (if discrete functional cell types existed, we’d see an ARI peak at some cluster count—not a decaying trend, as we saw in point 4 for the marmoset retina). We hope this reply helps to clarify our contributions,methodology, and revisit the evaluation.

References:
[1] Weis et al. (2025) doi: 10.1038/s41467-025-58763-w
[2] Tong et al. (2023) doi: 10.1101/2023.11.03.565500
[3] Weiler et al. (2023) doi: 10.1093/cercor/bhac303
[4] Zhao et al. (2020) doi: 10.1038/s41598-020-60214-z
[5] Höfling et al. (2024) doi: 10.7554/eLife.86860
[6] Baden et al. (2016) doi: 10.1038/nature16468

Thank you for your detailed feedback, acknowledging that we are solving an important problem on a “comprehensive” real-world dataset.

We believe that the concerns raised are mostly due to misunderstandings. Let us first address the criticism (weaknesses) you raised:

• Suboptimal Training Strategy / Clustering Algorithm:
  We do train jointly. After a brief pretraining phase, the clustering loss is combined with the Poisson loss, similar in spirit to IDEC [1], thereby preserving local structure. To avoid biasing neuronal structure in any way we initialize neuronal embeddings as a constant at the beginning. Therefore, the model first needs to learn minimal feature separation for clustering to be meaningful. Additionally, IDEC also “pretrain a stacked denoising autoencoder before performing clustering”. Hence, we are already following your suggestions.
  Different optimal amounts of pretraining epochs across datasets are more likely explained by different model architectures and very different nature of data in different datasets (electrophysiology vs calcium imaging, hundreds of neurons to thousands of neurons in a model and different signal-to-noise ratios).

• Discovering Number of Clusters:
  We would like to clarify why ARI is suitable to detect the number of clusters: ARI measures the agreement between two partitions of points by evaluating whether pairs of points are assigned to the same cluster in both partitions. When a ground-truth number of clusters exists, the ARI should peak at that value because points are consistently grouped together. This makes our clustering approach well-suited even in scenarios where the true number of clusters is not known in advance. We validated it on a marmoset retina, where N=4 is the ground truth number of clusters and we indeed see a clear ARI peak at 4 clusters. | Clusters | ARI (mean ± std) | |----------|------------------| | 2 | 0.78 ± 0.07| | 3 | 0.52 ± 0.31| |4 | 0.95 ± 0.01| | 5| 0.78 ± 0.08| | 6| 0.77 ± 0.15| | 7| 0.81 ± 0.08| | 8| 0.85 ± 0.05| | 9| 0.79 ± 0.04| | 10| 0.85 ± 0.07| | 15| 0.71 ± 0.07| | 20| 0.74 ± 0.06| | 25| 0.67 ± 0.04| | 30| 0.49 ± 0.02| | 40| 0.41 ± 0.04|

  We will integrate this analysis in the final version of the paper.
  As you suggested, BIC could be an alternative way to select the number of clusters. As BIC requires held-out neurons to estimate the test likelihood, we did it on mouse V1 datasets (marmoset retina is too small for it as it has ~150 neurons). For each mouse we randomly hold-out 200 neurons for each of the seven mice, which were trained together with the model but were ignored during the clustering procedure which fitted the TMMs. We used a model with 10 pretraining epochs and β=10^4. After the model was trained we a) computed the likelihood on the hold-out neurons and b) performed an additional E-step on the held-out neurons, calculating the soft assignments, such that we could evaluate ARI on only these 200 neurons as well.
  We computed BIC as $BIC = -2 log L + (2 \cdot k \cdot d + 1) \cdot log(N)$, $N$ is the number of samples ($N = 200 * 7$), $d$ is the dimensionality of the embeddings ($k=128$ in our case), $k$ is the number of clusters and $log L$ is the likelihood computed as $log L = \log\left( \frac{1}{K} \sum_{k=1}^{K} t(x \mid \mu_k, \mathrm{var}_k, \nu) \right)$. As we have not observed any significant changes in the likelihood, BIC just linearly grows with the number of clusters (number of model parameters), indicating the lack of clear cluster peak, agreeing with ARI (Fig. 4Hin the paper).

  Clusters	Likelihood	BIC
  5	375.3 ± 27.8	8529 ± 55.6
  10	348.5 ± 15.7	17855 ± 31.5
  15	355.0 ± 34.4	27115 ± 68.7
  20	335.1 ± 3.2	36427 ± 6.3
  25	382.9 ± 32.2	45605 ± 64.4
  30	387.2 ± 42.7	54869 ± 85.3
  35	351.2 ± 30.2	64213 ± 60.5
  40	371.2 ± 33.4	73446 ± 66.8
  45	357.4 ± 26.6	82746 ± 53.2
  50	367.7 ± 22.1	91998 ± 44.3

• Insufficient Baselines: We use post-hoc GMM/k-means clustering baselines due to the lack of more suitable alternatives. Our method already resembles IDEC as we do joint optimization to preserve local structure. We cannot use vanilla DEC since it forces unit-scale embeddings, but our embeddings are model weights whose scale is determined by the scale of neuronal activity. Thank you for pointing us to the “You Never Cluster Alone” paper [2]. While interesting, its contrastive approach would require augmentation of model weights—our per-neuron embeddings. As it’s not clear how one would do data augmentation in this case, this would constitute a separate research project.

• Low predictive performance: The predictive performance is not low – it’s close to the state of the art. It just reflects that predicting neuronal responses in mouse V1 is a hard task. Our numbers are comparable to the best models on Sensorium 2022 [3] – Table 3. A few percent can be gained by ensembling multiple models. We show the potential for this improvement by ensembling models across 8 seeds - this gave us 41% correlation for models with 5, 10 and 20 clusters (there were 8 seeds for each number of clusters, we did not ensemble across a number of clusters). Please note that this is a qualitative comparison as sensorium 2022 live and final test sets are not publicly available, we used test sets from 5 fully publicly available mice to compute the test metrics.

• Supplementary Materials: We apologize for the confusion. We intended to upload all supplementary materials together with the main text to preserve hyperlink functionality, with a single plot being added later. We will integrate all materials into a single document.

With respect to your questions:

1. Mode collapse: To clarify: we are not addressing general DEC mode collapse, but adapt DEC to our specific setting, where embedding scales are not arbitrary. In standard DEC, unit-scale clusters are acceptable because the latent space of the autoencoder can adapt. In our case, the embeddings to be clustered are model weights. Their scale is constrained by the scale of neuronal activity. Changing their magnitude would harm the predictive model. Therefore, we need to adjust the scale and shape of the clusters. To do so, we introduce scale matrices that let the model capture anisotropic cluster structure and non-unit scale. As noted in line 159, these matrices are initialized from empirical within-cluster variances, computed after a few pretraining epochs using only the Poisson loss—providing a data-driven, meaningful starting point.

2. Batch effects: As you correctly noticed, we do train several models for mouse and marmoset retinas. We do it due to the differences in the experimental conditions (presumably different temperature) which influences the speed of the responses so much that this would be the most trivial dimension to separate the neurons for different groups [4, 5]. While several works try to address this issue and integrate several retinas into the same model [6, 7, 8], this is still an active area of research, which is not the main focus of our paper. We also would like to note that DECEMber is not a specific model per se, but rather an additional loss that we integrate with various models. This also means that models for monkey V4, mouse V1 and mouse retina all have slightly different architecture, especially on the readout layers. And for mouse V1 and monkey V4, we do train jointly on several animals, as the experimental conditions between subjects there are comparable and DECEMber does not suffer from batch effects there.

We will add IDEC[1] into the related works section and integrate the BIC analysis to the appendix.

We hope these comments shed some light on our motivation, technical and design choices. We also hope that it would help you to reconsider some criticism.

References:
[1] Guo, Xifeng, et al. "Improved deep embedded clustering with local structure preservation." Ijcai. Vol. 17. 2017. [2] Shen, Yuming, et al. "You never cluster alone." Advances in Neural Information Processing Systems 34 (2021): 27734-27746.
[3] Willeke, K. F., Fahey, P. G., Bashiri, M., Hansel, L., Blessing, C., Lurz, K. K., ... & Sinz, F. H. (2023, August). Retrospective on the SENSORIUM 2022 competition. In NeurIPS 2022 Competition Track (pp. 314-333). PMLR.
[4] Zhao, Z., Klindt, D. A., Maia Chagas, A., Szatko, K. P., Rogerson, L., Protti, D. A., ... & Euler, T. (2020). The temporal structure of the inner retina at a single glance. Scientific reports, 10(1), 4399.
[5] Fritsches, K. A., Brill, R. W., & Warrant, E. J. (2005). Warm eyes provide superior vision in swordfishes. Current Biology, 15(1), 55-58.
[6] Höfling, L., Szatko, K. P., Behrens, C., Deng, Y., Qiu, Y., Klindt, D. A., ... & Euler, T. (2024). A chromatic feature detector in the retina signals visual context changes. Elife, 13, e86860.
[7] Vystrčilová, M., Sridhar, S., Burg, M. F., Gollisch, T., & Ecker, A. S. (2024). Convolutional neural network models of the primate retina reveal adaptation to natural stimulus statistics. bioRxiv, 2024-03.
[8] Gonschorek, D., Höfling, L., Szatko, K. P., Franke, K., Schubert, T., Dunn, B., ... & Euler, T. (2021). Removing inter-experimental variability from functional data in systems neuroscience. Advances in Neural Information Processing Systems, 34, 3706-3719.

Thanks for engaging in the discussion!

Suboptimal training: We are happy to clarify more:

• Good clustering depends on meaningful initialization without it, we risk forcing points into incorrect groups due to early guesses, leading to (random initialization) biased results. That is why both DEC and IDEC also include pretraining (IDEC page 3, sec. 3.1 “Follow suggestions in [Xie et al., 2016], we also pretrain a stacked denoising autoencoder before performing clustering. After pretraining…”). While we understand concerns about pretraining length affecting results, this limitation applies to all such methods. We are simply more transparent about it. Importantly, predictive performance remains stable across pretraining durations, so reproducibility is unaffected. However, internal representations are sensitive to training details (e.g., learning rate), so only models trained under the same conditions should be compared.

• “It is unclear to what extent the learning remains "joint": We will clarify in the main text that for best results, β just needs to roughly balance the clustering and Poisson losses, which can be estimated with a short (1–2 epoch) run. Both losses improve significantly (clustering: $10^8 → 10^6$, DECEMber: $5×10^7 →3×10^7$, same as without clustering loss), along with both ARI and correlation improvements, indicating proper optimization for both parts. Whether the losses help or hinder each other is an intentional feature of DECEMber—it tests if clusters truly exist. On mouse V1, increasing β to the point where performance drops shows that forcing clusters can erase functional structure without improving consistency. This is not a weakness of the method—it reflects the data’s continuous nature. Regarding data modalities, our paper includes experiments on three distinct types—micro-electrode array (marmoset retina), extracellular spikes (monkey V4), and calcium imaging (mouse retina and V1)—which vary widely in time resolution (kHz vs Hz) and signal-to-noise ratio. We also show in our response to reviewer mk7Y (signal-to-noise ratio section) that DECEMber remains robust even in low SNR settings.

Distinction from IDEC: In DEC and IDEC the same samples are encoded and clustered. In our case, we cluster regression weights, not input latents. The model maps images $x_i$​ to responses $y_j$ via $y_j(x_i) = \phi(x_i) * w_j$​; here, $x_i$ are regression samples, while $w_j$ (not $\phi(x_i)$ as in DEC/IDEC) are the clustering samples. This setup differs significantly from DEC/IDEC and requires substantial adaptation of DEC/IDEC—adding EM updates and a learnable scale matrix. DEC/IDEC serve more as inspiration than direct baselines. Moreover, in our toy example, DEC fails to separate nearby clusters, while DECEMber succeeds. And as shown in our reply to reviewer mk7Y, combining Poisson loss with DEC (e.g., IDEC) still leads to collapsed clusters, whereas DECEMber correctly separates them even in noisy settings.

Contrastive methods: Thanks for the suggestion—we’ll include it in the discussion.

BIC: Thanks for the BIC suggestion we will include it to the appendix. We are sorry, we found an error in our previous log likelihood and BIC calculations - here are the correct values on mouse V1 computed on ~53,000 neurons used in the training including models with 2-5 clusters, as you asked. Due to the large dataset, log-likelihood values are very large, causing BIC to decrease monotonously with more mixture components. Similar to ARI, BIC shows no clear peak to indicate an optimal cluster number.

Mouse V1

However, we argue that ARI is a more sensitive and suitable metric, even when the number of clusters is unknown. In our original reply in ”Discovering Number of Clusters” section, for marmoset retina ARI shows a clear peak at 4 clusters—the known ground truth, while for BIC (table below), there is no clear peak at all, the optimal values are 2 or 3, not 4 as expected.

Marmoset retina

We are sorry about the typo above, it should be ”$d$ is the dimensionality of the embeddings ($d=128$ in our case), $k$ is the number of clusters”.

Thank you for your positive review, appreciating that our method addresses a “question of broad importance in neuroscience” and can be “combined with a broad class of models”.
Below we are happy to address your questions and comments from the 3rd, 4th and 5th paragraph:

• Toy model from the manuscript:
  You say, “the neurons in the toy model are linear (and deterministic if I understand correctly?”.
  Yes, you are correct that the toy model is linear and deterministic. Its primary purpose was not to test for robustness to noise, but to create a minimal setting where we could clearly demonstrate a specific failure mode of the original Deep Embedding Clustering (DEC) loss. The vanilla DEC loss fails because it assumes a fixed unit scale for the clusters, causing the centroids to collapse. Our toy example shows how introducing a learnable scale matrix Σj is essential to solve this problem.

• Signal-to-noise ratio:
  You also suggested: “In the case of the toy model, what would adding a stochastic term to the neuron outputs do to the results? How would they change by the SNR of the resulting neurons?”
  As you correctly pointed out, its signal-to-noise ratio is a very important aspect of the data and as you suggested, we added a stochastic term in the toy example to address it.
  Specifically, we generated:

  • 1100 of images $I$, where each image is a 128-dimensional vector drawn from a standard Gaussian: $x_i \sim \mathcal{N}(0, I_{128})$. We do a train/test split with a train set containing 90% and the test set containing 10% of the data.
  • 1000 of neurons $N$, where the neuron weights are drawn from two Gaussian distributions to form uneven clusters: first “cell type” as $w_j \sim$ $\mathcal{N}($0$, I_{128}), \text{for } j = 1, \dots, 700$, and second “cell type” as $w_j \sim$ $\mathcal{N}($1$, I_{128}), \text{for } j = 1, \dots, j = 701, \dots, 1000$
    The clean neural responses were simulated as a dot product between the images $I$ and neurons $N$. To simulate noisy observations, we added Gaussian noise independently for each neuron and image: $\boldsymbol{\varepsilon} \sim \mathcal{N}(0, \sigma^2)$. The noisy responses are thus given by: $y_{ij} = w_j^T x_i + \varepsilon_{ij}$.
    Since both the input $\mathbf{I}$ and the noise $\boldsymbol{\varepsilon}$ are mean-centered, the Signal-to-Noise Ratio of neuron $j$ (SNR_j) is defined as: $\mathrm{SNR}_j =$ $\frac{\operatorname{Var}_i(w_j^T x_i)}{\operatorname{Var}_i(\varepsilon_ij)}$ (both $i$ and $j$ are indexes for $\varepsilon$ but openreview struggles to display it correctly).
    In this setup, we vary the SNR by adjusting the noise variance $\sigma^2$, thereby controlling the noise power in the simulation.
    We varied the SNR between 0.001 and 10 and same as in the paper toy example we first pretrained the regression model on MSE for 30 epochs and then finetuned with DEC or DECEMber + MSE loss for 150 more epochs. We calculated the Pearson correlation on a left out test set. Both DEC and DECEMber converge to similar performance since the MSE drives the learning of the model’s weights. Below you can find the table with the distance between cluster centers given different SNRS (the ground truth distance between the cluster centers is $\sqrt{128} \sim 11.3$). We see that even for low SNR DECEMber successfully separates the clusters, though the distance is not ideal before SNR is above 1. However, for DEC cluster collapse happens independently of SNR. Since the weights (e.g. neurons) are generated based on predefined clusters (means of the Gaussians), we know the ground truth label for each neuron. To evaluate the performance of DEC and DECEMber we measure the proportion of neurons that are assigned to the correct cluster.
    |SNR|Distance between cluster centers||Predictive accuracy (correlation)||Cell classification accuracy|| |---|---|---|---|---|---|---| ||DEC|DECEMBER|DEC|DECEMBER|DEC|DECEMBER| |0.001|1.968|3.908|0.000|0.000|51.6|50.1| |0.01|1.213|4.370|0.022|0.025|61.0|79.0| |0.05|0.197|6.993|0.100|0.109|79.0|98.1| |0.1|0.048|8.828|0.180|0.189|59.2|99.8| |0.5|0.015|10.494|0.489|0.494|51.4|100| |1|0.007|10.76|0.641|0.642|52.6|100| |2|0.011|10.95|0.771|0.771|50.2|100| |5|0.010|11.058|0.890|0.890|51.5|100| |10|0.015|11.075|0.941|0.940|50.3|100|

• Relationship Between Model Performance and Clustering:
  You asked “Can you identify a systematic relationship between clustering and model performance?”, which is a very interesting point.
  You are right, “conceptually, if we haven’t reasonably captured the appropriate set of features to describe the neurons, then clustering should be a lost cause”. E.g. on a toy example for the smallest SNR=0.001 the cell types accuracy is worse than a random guess (the probability that a random guess matches the true label $P(\text{predict}=i) \times P(\text{true label}=i) = 0.7^2 + 0.3^2 = 0.58$). However, even the minimal catch of meaningful signal already significantly improves the clustering accuracy for DECEMber (SNR=0.01 gives 2.5% correlation and 79% cell types classification accuracy). And of course, better model performance helps to improve the clustering: On the toy example above we see the cell types classification accuracy improving up to SNR=0.5, where it saturates as it reaches the best possible score.
  The models we use for real data, including mouse V1, are close to the state-of-the-art [2] and actively used for closed-loop [3] or in-silico experiments [4,5]. Therefore, we believe they do capture meaningful features even though the correlations are far from 1. The fact that DECEMber improves cluster consistency on the very noisy mouse V1 dataset provides strong evidence that DECEMber is effective in the high-noise, low-SNR regime that is typical for in-vivo neuroscience data, even though the models are not perfect.
  You also state that “the strength of a clustering approach is the potential to share information across many neurons that have similar properties.” That’s correct only if the clusters are very homogeneous. If there is a lot of within-cluster variance, there is not so much information to share. Pulling all neurons towards the cluster mean would actually hurt the performance, because the neurons indeed are different.
  You also mentioned in your summary that “imposing the clustering prior appears to hurt performance of the models.” It doesn’t actually hurt performance unless we overdo it. For smaller beta~=10^4 (Fig. 4H), predictive performance is maintained but embedding consistency improves substantially. The experiment of increasing the clustering weight, β, to the point where performance drops is intentional. It demonstrates that aggressively forcing a clustered structure can erase functionally relevant details, but does not improve embeddings’ consistency. This result supports our conclusion that excitatory neurons in the mouse visual cortex likely form a continuum rather than discrete types. It is not a weakness of the method – it is a property of the data!

• $\nu$ as a Hyperparameter:
  In the 5th paragraph, you ask if there is a reason we ”treat $\nu$ (in the t-distribution) as a hyperparameter”.
  While it is possible to optimize ν in a t-mixture model, it does not have a simple closed-form update in the M-step, unlike the mean and scale matrices. We tried both having as a learnt hyperparameter on toy data and also explored the values between 2-20 on mouse V1 data, however, it had little to no impact on clustering outcomes or overall model performance, which is consistent with the original DEC paper [1], so we chose to fix $\nu$ as a hyperparameter for simplicity and computational efficiency. We will add a section in the appendix summarizing these findings.
  |ARI ± std|clusters|$\nu$| |---|---|---| |0.416 ± 0.012|5|2.1| |0.417 ± 0.013||5| |0.418 ± 0.013||10| |0.42 ± 0.012||20| |0.299 ± 0.025|10|2.1| |0.299 ± 0.025||5| |0.301 ± 0.025||10| |0.303 ± 0.027||20|

• Line 140: yes, “feature i” is equivalent to “neuron i”

We thank you for the biologically meaningful questions and helping us to improve our toy model. We will integrate it into the final manuscript.
We are happy to address any follow-up questions as well.

References:
[1] Xie, J., Girshick, R., & Farhadi, A. (2016, June). Unsupervised deep embedding for clustering analysis. In International conference on machine learning (pp. 478-487). PMLR.
[2] Willeke, K. F., Fahey, P. G., Bashiri, M., Hansel, L., Blessing, C., Lurz, K. K., ... & Sinz, F. H. (2023, August). Retrospective on the SENSORIUM 2022 competition. In NeurIPS 2022 Competition Track (pp. 314-333). PMLR.
[3] Walker, E. Y., Sinz, F. H., Cobos, E., Muhammad, T., Froudarakis, E., Fahey, P. G., ... & Tolias, A. S. (2019). Inception loops discover what excites neurons most using deep predictive models. Nature neuroscience, 22(12), 2060-2065.
[4] Ustyuzhaninov, I., Burg, M. F., Cadena, S. A., Fu, J., Muhammad, T., Ponder, K., ... & Ecker, A. S. (2022). Digital twin reveals combinatorial code of non-linear computations in the mouse primary visual cortex. BioRxiv, 2022-02.