Range, not Independence, Drives Modularity in Biologically Inspired Representations

We thank the reviewer for their attention and reading of our paper. Below we try to address the concerns they raised.

Weakness 1: Contains too much!

Weakness 2: Figure and text panels are small throughout

We agree that the paper is full and apologise for any lack of clarity that results. To address this, where possible, we have increased the size of figure labels and figures throughout the paper. Further, we have moved the PFC results to the appendix, making the focus of the neuroscience results more in line with the main thrust of the paper, and more coherently centred on the entorhinal cortex. With this extra space we have significantly expanded various parts of the paper, in particular the section discussing the main theorem and providing intuition and interpretation. We hope these changes have somewhat improved the clarity.

Weakness 3: How relevant are extreme points of distributions for actual neuroscience?

The reviewer is right. Our results depend on the extreme points in worrying ways. The extreme points are determined by outliers, and it seems strange for very unlikely outlying points to govern the behaviour of the optimal representations.

The reason this happens in our theory is that we require the representation to perfectly decode the labels for all datapoints. If, instead, the reconstruction loss is included as a term to be minimised alongside the activity and weight energy then this failure mode would be removed. Instead of being forced to encode all outlying points, the representation could choose to ignore an unlikely extreme point, paying a small reconstruction cost for a saving in energy.

We would like to theoretically understand this effect, however each of these theoretical developments takes time and we have not yet been able to develop this one. We are now working on empirical results to show this effect that we hope will be ready before the end of the discussion period, and if not by then, definitely by the time of the conference.

That said, the idea that what matters for modularisation is that the support of variables is independent even if the distribution is not, seems a more reasonable conclusion than alternatives for actual neuroscience. For example, testing whether two variables are statistically independent is difficult, but checking whether they are extreme-point independent just requires tracking whether you’ve ever seen the four corners of the distribution occur. Indeed, with range-independence we are able to explain neural data that no other theory can (figure 4). We are in the process performing neural experiments in rodents to test these conclusions, though of course this is beyond the scope of the present work.

Continued - neurons do not autoencode: Question 3: Autoencoding is a limited objective, what’s up with it?

This is certainly true. Neurons are doing far more than autoencoding. However, our goal here is not a particularly accurate model of any neural circuit performing a computation. Rather, we seek a simple model of mixed selectivity vs. modularity, and why neurons might choose one or the other. A minimal model has to force the neurons to encode the variables somehow. This could be done by making the neurons perform computations with those variables, or by producing a particular behaviour as the reviewer suggests. But then theoretical analysis will likely be hard and dependent on the simplified task that was studied. Instead we try to make minimal assumptions, and just study neurons forced to encode variables. We chose autoencoding because it doesn’t make any assumptions about what the variables are being used for, just that they are represented and decodable (linearly for theoretical analysis, empirically we drop this). That this simple setup is sufficient to produce interesting behaviour that matches neural responses further justifies this choice in our mind. This of course does not negate the fact that ideally we would study theories of neurons doing meaningful computations, and this is a target for our future work. We have included a sentence in the discussion to this effect:

Additionally, our model is a purposefully simple model, there are many biologically pertinent effects whose impact on modularity would be interesting to study, such as connection sparsity, anatomical constraints, noise, or requiring additional computational roles from the network.

Thank you again for your valuable feedback. As the end of the discussion period is approaching, we are hoping to hear your thoughts on our response. We hope that we have addressed your comments, and we would greatly appreciate it if you were willing to consider increasing your score if you are satisfied with our response.

We would like to thank the reviewer for their careful, and largely positive, review of our work. Below we hope to address some of their concerns.

Weakness 1: Non-negative activities, what about inhibitory neurons?

The reviewer is right that in our simple model we do not split between excitatory and inhibitory neurons. We do this as we are interested in the simplest possible account of modularity and so only use the fewest constraints. We see this as a real positive. Thus our theory is largely modelling how excitatory neurons (cortical pyramidal neurons) should encode variables as simply as possible. With that in mind, it is interesting that by focusing on excitatory neurons we can already get non-trivial results that match neural data. Nevertheless, we agree with you that incorporating inhibitory neurons may lead to further non-trivial situations in which modularity occurs - we leave this for future work.

Weakness 2: L2 is cool, but why no other biological constraints (sparse connectivity, synaptic range…)?

This is a good question. Ultimately we are looking for the simplest set of biological constraints that lead to modularity. You’re right that to have the fullest picture of how biological constraints interact with representation, then including things like sparse connectivity and synaptic range would be great. But again, we’re looking for the simplest set of constraints that induce modularity. It is surprising that simple energy minimisation in a linear autoencoder got us this far, yet was quite hard for us to understand fully. Further understanding things like sparse connectivity and synaptic range is in our overall mission, but that is a big ambition and will be other papers. We now include a statement in the discussion to this effect:

Additionally, our model is a purposefully simple model, there are many biologically pertinent effects whose impact on modularity would be interesting to study, such as connection sparsity, anatomical constraints, noise, or requiring additional computational roles from the network.

With particular reference to the use of L2 activity loss (rather than other reasonable choices), in our new submission we include an additional theoretical result that shows that range-independence is sufficient to drive modularity for a wide range of choices of activity norm (new appendix C).

Weakness 3: How could the theory extend to more complex datasets, e.g. images and text

We would likely proceed much as we did in the last panel of figure 2, which is an application of our theory to a complex image dataset. Apologies if this was not clear, we now highlight this in the main text:

We study the performance of QLAE trained to autoencode a subset of the Isaac3D dataset (Nie 2019), a naturalistic image dataset with well defined underlying latent dimensions.

So to proceed we would assume that the measured representation is encoding a particular set of latent variables, then reason about how the theory suggests the distribution of those sources latents might lead them to be optimally encoded in a modular or mixed fashion. This has many obvious weaknesses (what are the latents? What if it is not just encoding but is computing with the latents, won’t that affect things?), but presents a good start that is already predictive, as in figure 2.

Weakness 4: Why do the PFC results not match? Both in subspace angle and neuron alignment

Yes, this is confusing. We have a lot of ideas for exploring this discrepancy (in brief: the numerical subspace alignment might be fixed by changing hyperparameters of the task; and we are diving into the original neural data to test some hypothesis about what might cause the mixing on a neural level). These directions require more work and, and so we have decided to move this section to the appendix and replace it with a discussion of other potential causes of mixed selectivity, beyond the currently predominant theories that our work suggests. Thank you for raising these issues.

Question 1: Does the theory generalise to other brain areas?

Despite moving the PFC section to the appendix, we remain hopeful that our work applies broadly to neural coding. We are interested in testing it in sensory and motor areas, as well as refining the removed PFC results. It is known that cells in parietal cortex, for example, that neurons are highly responsive for single tasks (Lee, Krumin, Harris, Carandini, 2022) and not others (they are modular), while in the prefrontal cortex there are modular neurons for things like the abstract structure of a sequence (Shima & Tanji 2007). However, in this study, and most other neuroscience studies, they do not have diversity in their task structure to test for concepts like range independence. We are actively collaborating with experimentalists to run studies which directly test our theoretical claims in a variety of brain regions, and we hope to be able to answer this question in the coming years.

Thank you again for your valuable feedback. As the end of the discussion period is approaching, we are hoping to hear your thoughts on our response. We hope that we have addressed your comments, and we would greatly appreciate it if you were willing to consider increasing your score if you are satisfied with our response.

We thank the reviewer for their careful and detailed review of our work that helped us significantly improve it. Below we hope to address some of their concerns.

Weakness 1: Writing could be improved, including theorem introduction in theorem 2.1

We’re sorry for the unclear writing.To remedy this we have significantly changed the buildup to theorem 2.1 by adding additional details to section 2.2 clarifying the intuition that can be used for why range independence leads to modularity, which now reads:

The key takeaway lies in how $b_j$ is determined by a joint minimization over $s_1$ and $s_2$ (5). Assuming positive $w_{j1}$ and $w_{j2}$; then if $s_1$ and $s_2$ take their minima simultaneously, as in the middle row of Fig1a, then mixed bias must be large:

$

b_j = -\min_{s_1, s_2} \left[ w_{j1} s_1 + w_{j2} s_2 \right] = - \min_{s_1}[w_{j1} s_1] - \min_{s_2} [w_{j2} s_2] = b_{j'} + b_{j''}

$

And the energy of the mixed solution will always be worse than the modular, since $b_j^2 = (b_{j'} + b_{j''})^2 > b_{j'}^2 + b_{j''}^2$. Alternatively, mixing will be preferred when $s_1$ and $s_2$ do not take on their most negative values at the same time, as in the bottom row of Fig 1a, since then $b_j$ does not have to be as large to maintain positivity, and the corresponding energy saving satisfies the key inequality (6).

Further, we have added the following clarifying sentences after the introduction of theorem 2.1 to link it to the previously discussed intuition.

These inequalities come from the difference in activity energy between a modular and mixed solutions, just like the intuition we built up in Section 2.2, and in particular Equation 6.

More broadly, we have moved one of the neural analysis sections to the appendix and used the space to expand the other sections. We have expanded the discussion of the effects of range independence (new section 2.4), grown the figure labels to make it clearer, and rewritten the mixed selectivity part of the discussion. Our most pertinent changes were to figure 1, in answer to another of your comments:

Weakness 7: Unclear what’s going on in figure 1, came out too early

Apologies for Figure 1. We have now completely re-worked it, moved it much later in the paper, and added a much extended caption to try and more clearly explain it. The new intuitive figure (new fig 1A) focuses on the difference between a mixed encoding of a range-independent and range-dependent pair of variables, following more closely the discussion in the intuitive section. Further, we have made it larger to make it clearer, and added to the caption to walk the reader through our argument in the sketch figure (1a), and the definitions of all quantities in the data figures (1b and 1c). We thank you for pointing to these shortcomings.

Weakness 2: Applications to data appear to be preliminary

Weakness 3: Non-neuron aligned subspaces in PFC, what’s going on there?

Relatedly: Question 2: What is the relationship between modularity and orthogonality?

We agree with the reviewer, in the original submission the PFC section was slightly too preliminary, and we merged orthogonality and modularity too quickly. As such, we have moved these results to the appendix and replaced it with a section that focuses on how our theory impacts interpretations of the role of mixed selectivity from data, which we would be grateful if the reviewer considered. We thank the reviewer for prompting us to think more carefully about this, and we are currently running further analysis to address their and other concerns.

However, regarding the entorhinal mixed selectivity results, we disagree with the reviewer and consider them thorough. We hope that our response to your next question will convince you of the thoroughness of the results.

Thank you again for your valuable feedback. As the end of the discussion period is approaching, we are hoping to hear your thoughts on our response. We hope that we have addressed your comments, and we would greatly appreciate it if you were willing to consider increasing your score if you are satisfied with our response.

To briefly explain our final submission edit, we fixed a few typos, and changed some task parameters in order to make the PFC ersult numerically align with the data.

How aligned the two subspaces are in our PFC model networks depends on critical hyperparamters in the theory such as the weighting of weight vs. activity loss, and the length of the delay period. We played a little with these and found a set that agreed numerically with the data. The crucial prediction remains the same however: if the stimuli are range-independent then, regardless of these choices of hyperparameters, the subspaces are orthogonal. If they are range-dependent then the subspaces can align, and how much they do depends on hyperparameter choices in our theory.