Disentangling the Roles of Distinct Cell Classes with Cell-Type Dynamical Systems

We thank the reviewer for their positive assessment of our work. We are glad that the reviewer found our work to be well presented, and our experiments to be comprehensive. Below are out detailed responses to the reviewers questions:

1. Clarifications on figures 2D / 3CD: Thanks for bringing these to our attention, we will add detailed captions to clarify these figures. In Fig 2D, the y-axis represents the magnitude of the eigenvalues. In Fig 3CD, the dashed line shows 0% bias, which means that the animal makes perfect choices. In Fig 3D, the stars indicate that the experimentally observed ipsilateral biases during both early-half and later-half perturbations in the ADS are significantly above 0% (Yartsev et al. 2018). In Fig 3C, the observed bias is only significant during later-half FOF perturbations as indicated by the star, and not during the early-half perturbations. Furthermore, the difference between the two perturbation effects in FOF is significant (Hanks et al. 2015).

2. E latents in ADS: Thanks for pointing this out. ADS does not have E neurons, thus we thought it was not important to visualize E latents in ADS. We discuss this in sec A3 of the appendix, but will make sure to mention this in the main text to avoid confusion.

3. Extension to other cell types: Thanks also for this comment; we feel this is an important point about the generality of our model, and we have elaborated on this in more detail in our response to all reviewers.

4. Performance on other brain regions and tasks: We focused on one task in this work, and fitted our models to one dataset containing FOF and ADS neurons. However, we would like to point out that our results (Fig 3CD) are consistent with two other papers (Hanks et al. 2015 and Yartsev et al. 2018), which study the same task but using distinct datasets. Our focus here is to introduce the model, and comprehensively demonstrate its merits using the poisson clicks task. We hope future work will apply CTDS to other tasks and brain regions.

5. Real-time decoding and perturbations: CTDS is indeed capable of real-time decoding. Once the model has been fitted, during inference Kalman filtering can be used to infer underlying latent states and behavior. Thus, it is also possible to extend this framework to control the inputs being provided to the model in a closed loop while running real-time decoding. Thanks for this important comment, we will add this to the discussion section.

We thank the reviewer for their positive assessment of our paper and constructive suggestions for improving it. We are glad that the reviewer found our experiments to be extensive, and our causal perturbation results to be impressive. However, we feel that the reviewer has misunderstood one of our key findings, as we explain below. We apologize for not making this point clearer, and would humbly request that the reviewer consider raising their score if indeed they are persuaded by our response (on that and several other issues that we describe below).

1. Comparison between CTDS and a standard LDS (Fig 2C, 3C): We would like to clarify the takeaway messages from both these figures, and want to emphasize that they do not suggest that LDS models outperform CTDS. Fig 2C shows a comparison between choice accuracy as decoded from CTDS and LDS models. Indeed, both models perform equally well, however CTDS provides greater interpretability by allowing us to characterize the information encoded by the different cell types. Furthermore, CTDS has higher test-LL which means it in fact captures neural activity better than LDS.
   More importantly, in Fig 3C (left panel), we show the classification error when perturbing FOF during either half of a trial. Experimental perturbation results from rats show that this error is low during early-half perturbations, but high during late-half perturbations (see Fig 3C right panel). CTDS is indeed able to capture this phenomenon due to the distinction between E and I latents, while a regular LDS does not recapitulate these findings. The fact that the “choice error” bar for the LDS model is lower than that of the CTDS model in Fig 3C does not mean that the LDS model is performing better — in fact, the opposite is true, since the point of this figure is to show that CTDS model captures the qualitative pattern of errors found in real behavior experiments (from Hanks et al. 2015, shown in the right panel of Fig 3C) better than LDS. We apologize for the confusion on this point, and we will rewrite the figure caption and text to make this clear. This is a key contribution of our work, so we hope this alleviates the reviewer’s concern.

2. Comparisons to nonlinear RNNs, incorporating non-linearity: Our rationale for showing an equivalence between CTDS and linear RNNs was to illustrate that CTDS acts as a bridge between mechanistic and descriptive models. That is, low-rank linear E-I RNNs can be exactly transformed to a CTDS model. We agree with the reviewer, however, that non-linear RNNs are more expressive. In future work we would certainly like to extend CTDS to incorporate nonlinearities, which we agree would make it both more expressive and more biologically plausible; we will certainly add this point to the Discussion. However, we feel that the idea of incorporating cell types into LDS models, along with the empirical results on perturbation experiments (and the mathematical connection to linear RNNs) nevertheless make for a worthwhile contribution in their own right, which we hope the reviewer will agree on.

3. Evaluating the robustness of the model using incorrect cell-type information: This is a great question, and we thank the reviewer for bringing this up. As the reviewer points out, it may be difficult to brain accurate cell-type information. Sec 5 of our paper which focuses on inferring cell type identities gets at this question: we mask the identities of up to 50% neurons in our dataset, and fit CTDS to these datasets while also inferring neuron identity. As shown in Fig 4, CTDS is able to infer neuron identity significantly above chance for up to 50% of neurons. To answer the reviewer’s question more directly, we have also attached a figure showing test log-likelihood of CTDS fitted to datasets with varying numbers of masked neurons, as compared to when we know all cell identities accurately (in the pdf attached with author rebuttal). As we can see, while test LL expectedly falls off as the percentage of masked neurons increases, it is robust when masking up to 20% of neurons. We will add this new result to our revised manuscript.

4. Relevant literature on inferring cell classes: Thank you for pointing us to these papers, we will be sure to cite and discuss them in the updated version of the manuscript.

5. Adding more fine-grained cell information: This is an important suggestion. Please see response to all reviewers for our comments on this.

We thank the reviewer for their detailed and insightful review. We are grateful that the reviewer found our work to be novel, technically solid, and of significance to the neuroscience community. Below are our responses to the reviewer’s comments and questions:

1. Relevant literature on sign constraints in neuroscience models: We agree with the reviewer that a large number of previous models have sought to incorporate Dale’s law and or sign-constrained weights. However, we would like to point out that past works have focused on incorporating Dale’s law in recurrent neural network models (RNNs), but NOT (as far as we are aware) into latent dynamical system models. For example, [Fisher et al. 2013, Haber & Schneidman 2022] have used sign-constrained RNNs to study properties of neural circuits, and [O’Shea & Duncker et al., 2022] have used them to study perturbation experiments . We have cited these works in Sec 3 of our paper, but will also mention them in the introduction for better contextualization per the reviewer’s suggestion. However, we are not aware of similar literature in the context of latent dynamical systems. Latent dynamical systems have typically focused on descriptive representations of neural activity, while sidestepping mechanistic properties of the brain. Our work, thus, bridges this gap by enforcing mechanistic constraints on linear dynamical systems. We thank the reviewer for bringing this up, and we will make this clear in the introduction of our revised manuscript.

2. Extension to other cell types, model nomenclature: We agree with the reviewer that here we only focus on E and I cell types, and have not discussed more fine-grain cell identities. However, we feel that our framework can easily incorporate multiple cell types within a single population or across multiple populations (as discussed in the example case of PV and SOM inhibitory neurons in the response to all reviewers). We apologize for not making this generality clear in our original paper, and will revise both the Introduction and Discussion sections to discuss specific ways in which CTDS can be applied to multiple cell types (and not just E and I).

3. Limitations section: Thanks for pointing this out. We will add an explicit limitations section in the manuscript discussing choice of latent dimension, and scaling to other tasks. Currently, we choose latent space dimensionality based on test log-likelihood as well as for ease of visualization / interpretation. However, our results qualitatively hold for a wide range of latent dimensions (we will add this in the appendix). In terms of applying the model to more naturalistic tasks beyond 2AFC, two main challenges would come up: (a) The stimulus could be high-dimensional: this would mean learning a correspondingly higher-dimensional B matrix in our current CTDS setup. Alternatively, we could also learn a lower-dimensional encoding of the stimulus, depending on the specifics of the task. (b) For naturalistic tasks spanning over larger timescales, a linear model might not be sufficient. In such cases, our approach can be extended to incorporate sign constraints in a non-linear setup such as a switching linear dynamical system.

4. Presentation tweaks: Thank you pointing these out, we have fixed these now.

Thanks for your comments and manuscript improvements.

I still feel that the nomenclature of "cell-types" is overselling a bit. The authors' global response points to PV and SOM as distinct types with specific connectivity/signs that future work could model. While connectivity and sign are indeed part of distinguishing these cell types, other properties are also important (e.g. dynamics of fast-spiking in PV vs. low-threshold spiking in SOM). As noted in my review, my concern was not that distinct connectivity/signs couldn't be represented in this framework (they can, as demonstrated in the model and multi-area variant). My concern was that other important functional properties that are widely accepted as important to the distinction of cell types (especially morphology and intrinsic dynamics) can't be trivially incorporated. So, the contribution here really is "sign-constrained latents". If renaming isn't desirable at this stage, then perhaps a discussion in the paper can highlight that this research direction is still wide open.

Overall, I am excited that this work explores how to make LDS models more mechanistic by incorporating biological properties, a contribution the authors have reiterated in their response. I am happy to maintain my score to accept.