Binding in hippocampal-entorhinal circuits enables compositionality in cognitive maps

Thank you very much for your thoughtful review and accurate summary of its main points. We concur that the strengths listed: theory (proofs and experimental validation) for every part of the RNS model, in addition to testable hypotheses for experimental neuroscience, are the core results of the paper. We would like to clarify a few points regarding weaknesses/limitations of the paper and answer your questions.

Code is not provided

We would like to politely mention that code to replicate experiments was included in the Supplementary Material zip of the submission.

There were several mention of compositionally as the motivation, but there is no direct analyses to show the effectiveness of model in compositionally

Thank you for raising this issue. We'd like to use this opportunity to more clearly define what we mean by compositionality and how the model achieves it.

We use the term 'compositionality' to denote a design principle of the HF model. Compositionality refers to complex representations that are composed by simpler building blocks and formation rules [A11]. This means that even a small number of primitives can be richly expressive, because they have a large combinatorial range. The representations we posit in the hippocampal formation meet this definition: they are composed by modules representing residue numbers and contextual tags that are formed by binding operations. These design choices result in the large and robust combinatorial range, as we show theoretically and empirically.

At the same time, we recognize that the term 'compositionality' has different meanings across machine learning, cognitive science, and neuroscience. We are also aware that there are interesting cases of compositionality that our model doesn't speak about. These include compositional generalization to out-of-distribution test datasets, and compositional generation of novel strategies, as studied in program induction.

To make these points explicitly, in the final version, we will include this definition of compositionality upon introduction of the term (around lines 19-20 in the current PDF), and we will discuss these other related senses of compositionality in the discussion under limitations (around lines 304-315).

There is virtually no comparison to other models in terms of coding range, robustness to noise and compositionality

Please see our discussion in the general rebuttal on model comparison. In that section, we have explicitly aimed to address the coding range, robustness to noise, and compositionality points of comparison.

We should also note that no other related work attempts to measure coding range (of modular attractor networks implementing RNS), robustness, and compositionality for a single model of the hippocampal formation. Consequently, we've done our best to indicate points of comparison when appropriate for individual experiments. We also feel that the theoretical foundations and interpretability of the model provide potential advantages over other models that could be proposed in the future.

How does the time-scale of unit responses compared to actual neurons? Is the attractor model fast enough to track the changes in the environment?

In our model, the unit responses are on the order of 100ms. We believe that this is a biophysically reasonable parameter, and it is consistent with the timescale of theta oscillations (4-12 Hz) believed to be important for neural computation in HF. Empirically, we find that this timescale is sufficient to track changes quickly enough for reliable path integration.

The potential prediction for encoding of episodic memory in this framework is not clear to me.

Thank you for raising this point. Page constraints made it a bit difficult to discuss this point in the main text, but we will gladly add this information to Sec. 4.2 and the Discussion in a revised version.

The larger picture is that many neuroscientists believe that the two functions of the hippocampus -- in episodic memory and in spatial navigation -- are supported by the same neural circuits and principles (e.g., [A12]). Put more starkly, spatial navigation and episodic memory are two sides of the same coin: one is navigation in real physical space, the other is navigation in a more abstract, conceptual space.

If this unified picture of navigation and memory is correct, then our model would have implications for how memories are structured and stored in HF. The experiments for sensory recall (Sec. 4.2, Fig. 7) are designed to explain how imperfect memories corresponding to sensory patterns can be denoised by the hippocampal-entorhinal loop. Further, in the Appendix (Sec. C.3 and Fig. S5), we test the model's ability to perform recall sequences of concepts, even in the presence of neural noise.

Regarding limitations:

Comparison to existing methods, or an ablation study on the proposed method to verify the intended computational role for each component

Please see our discussion of model comparison in the general rebuttal.

The ablation study that you mention is a helpful suggestion. In the PDF attached to the general rebuttal, we have conducted an additional ablation experiment (Fig. X3) to quantify the impact on performance.

[A11] Szabó, Z.G. (2022). Compositionality. The Stanford Encyclopedia of Philosophy.

[A12] Buzsáki, G., & Moser, E. I. (2013). Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nature neuroscience, 16(2), 130-138.

Thank you for your accurate summary of our work and fair assessment of its strengths, weaknesses, and limitations. We appreciate that you found the ideas to be interesting and comprehensive.

We agree that our modeling approach is "relatively high-level", and that such an approach comes with strengths and weaknesses. We would emphasize that our motivation for this abstraction is to include all of the components of the HF needed to have a minimal working algorithm, and no more. In other words, we can justify the functional relevance of each region and computational step, without introducing "bells and whistles" that might dilute that message.

In the general rebuttal to all reviewers, we have included some additional ways in which our model can be evaluated relative to neural data. These are meant to contextualize the predictions for neuroscience that are outlined in the discussion.

Regarding your specific questions:

Can you clarify what you mean by "carry-free" hexagonal coding?

Good question. Upon reflection, we will revise "carry-free hexagonal encoding" (lines 213-4) to "carry-free implementation of a triangular frame", in order to make the meaning of this statement clearer. For completeness, we also provide some further clarification below.

In general, carry-free means that the components of the representation can be updated in parallel, i.e., without dependence on the results of other updates. For example, binary representations are not carry-free with respect to addition, since the final state of a bit depends on two components and the results of computation ("carry-over") on lower-order bits.

For the triangular frame discussed in Section 3.5 and Appendix A.3, each 2-D position is represented with three coordinates. Each coordinate can take up to $m$ integer values from $\{0, 1, \dots, m-1 \}$. To illustrate why these computations are not carry-free in general, suppose we update coordinate [1, 1, 0] by [0, 0, 1] - the resulting coordinate is [1, 1, 1]. This is actually equivalent to the 2D position expressed by the coordinate [0, 0, 0], but we wouldn't know that without further computation. Thus, we'd either need further "carry-over" operations to reduce states to one member of an equivalence class, or we make equality testing cumbersome.

Can you reflect on scalability in terms of the number of neurons. For example, regarding the statement, "In particular, we require that distinct integer values are represented with nearly orthogonal vectors." how does this requirement affect the scalability of the approach?

We appreciate the question and opportunity to clarify.

In a $D$-dimensional space, linear algebra dictates there are only up to $D$ exactly orthogonal vectors. Our method relies on the fact that there are many more vectors that are almost orthogonal.. That is, they have a non-zero, but still small, inner-product. This idea is fundamental to the theory of dimensionality reduction in machine learning, and indeed, the kinds of random codes we employ here are commonly used for this purpose (e.g., [A7]).

Our analysis in Theorem 1 (Appendix A.1) implies that, to represent a set of $M$ distinct states using vectors whose inner-product is (with high probability) at most $\epsilon$, it suffices to take $D = O(\epsilon^{-2}\log(M))$. Note that the dependence of the dimension on $M$ (the total size of the universe) is just logarithmic! This result is consistent with a wide body of other work in the machine learning literature that obtains similar rates for coding schemes of this nature [A7, A8].

From a practical perspective, this means that a relatively modest number of neurons (as we use in our experiments) can achieve a large dynamic range.

What is the numerical precision required for numerical stability in terms of the neural activity and the synaptic weights? Can you discuss how realistic this approach is in the context of real neurons with firing raters under 100Hz?

This is an important question. We have a few comments:

• The von Mises noise experiments (in Figs. 3 and 4) provide an implicit answer, because adding phasor noise decreases reliability of the precision.
• To answer your question more explicitly, we have conducted a follow-up experiment testing the effect of quantizing to a small number of bits per synaptic weight. Please see Figure X2 in the PDF attached to the general rebuttal; we will also add it to the supplement). Within this range, we've found that a) 5 bits is nearly as good as full precision, b) even 3 bits still performs well, and c) higher precision faces diminishing returns, in a way that higher dimensionality does not. This last point is consistent with prior work on quantization in theoretical computer science (e.g., [A9, A10]), and with the observation that biological neural networks have high dimension but with low-precision components.
• Nothing in the model requires high firing rates to function.

[A7] Rahimi & Recht (2007), NeurIPS

[A8] Dasgupta & Gupta (2003), Rand. Struct. Algor.

[A9] Clarkson & Woodruff (2009), ACM STOC

[A10] Zhang, May, Dao, Ré (2019), AISTATS

Thank you for your thoughtful review. We appreciate your agreement about the rigor and suitability of the mathematical framework. Apologies if responses seem terse, we've tried to give clear explanations within the word count.

To make a better job ... What is it that main thing that it bring to the table? Apart from mathematical elegance.

Our work gives a theory of computations (representations and algorithms) in the HF. We posit that the HF instantiates our model's recurrent dynamics, population codes, and binding mechanisms.

These lead to testable predictions for experimental neuroscience and ascribe function to existing observations about neural data. The model's robustness (Figs. 3 & 4) could help explain the brain's robustness. We also list core contributions in the general rebuttal.

the paper conflates conceptual ... and computational/mechanistic similarity

We agree that the two are distinct. Our contributions to each are also distinct:

• Our model is conceptually similar because the five principles of spatial representation (outlined in Sec. 2.1) are the same for our model and (we believe) for HF. However, the intention is not to stop at mere conceptual similarity, but rather to motivate our proposed mechanisms.

• More significantly, the model has computational/mechanistic similarity. The variables and computations of the modular attractor network (detailed in Sec. 2.2) map onto specific neural populations and circuit-level dynamics. Sec. 2.3 maps the model's parts to neuroanatomy.

Binding and its realization in HF circuits is "at the core of potential achievements". We discuss binding further in the general rebuttal and re: question (4).

Thus, we don't think the paper conflates the two, nor is it exhibiting only conceptual similarity.

The evaluation ... better that some other ones?

Please refer to our discussion of model comparison in the general rebuttal.

Weaknesses:

The task ... was not clearly defined

These were defined in Appendix B: Experimental details. We run four categories of empirical tests; each described in a subsection of the appendix. Path integration is just one evaluation.

Still, we anticipate that the spirit of this comment is to make these clearer in the main text. We will definitely revise the main text accordingly.

a dissonance ... if the importance is not in that, then what is it in?

We respectfully disagree that the connections to biology are loose. Please refer to the discussion of computational/mechanistic similarity and response to question (2).

The evaluation of simulated trajectories...

Please refer to our responses for Question (3) / Figure 6A.

Questions: Re (1): The claim is the former: that the computations resemble those in the HF.

Re line 122: A grid module is a population of grid cells in MEC that have approximately the same "scale" -- the spacing between the firing field peaks that form the hexagonal lattice. Grid modules are seen as functionally significant since (a) scales appear discretized in experimental data, and (b) neuron's scale correlates strongly with anatomical location [9].

Re (2): There are two comparisons. We give examples of grid response fields from our model that resemble neural data (Fig. 6C, Fig. S1). Our model also recreates experimental data regarding global remapping (Appendix C.2, Figure S2). We would be excited to compare our model to further neural data, but we also believe that it is beyond the scope of this paper.

Re (3): a) It is a standard dataset used in computational neuroscience. Recent examples include [37, A6]. b) There are significant practical advantages: synthetic data gives more control to over room sizes, trial lengths, and numbers of trials. c) The dataset is realistic. The authors used statistics of actual rodent trajectories and validated simulation quality [36].

Re Fig. 6A (1): For the model without attractor dynamics, the update in equation 7 is replaced by $\hat{\mathbf{g}}_i(t + 1) = \mathbf{q}_i(v_t) \odot \hat{\mathbf{g}}_i(t)$.

Re Fig. 6A (2): Performance comes from the model's robustness to noise (Fig. 3, Sec. 3.3) and ability to interpolate between integer values (Fig. 4, Sec. 3.4). The task isn't trivial since noise accumulates over time and since the attractor network denoises sub-integers without using additional resources.

Re (4):

The predictions ... true in biological system?

Excitingly, we don't know yet! These predictions are offered as new hypotheses to test model similarity to the brain.

it would be great to have a more extensive explanation of what "multiplicative interactions ...

We appreciate the opportunity to clarify. Please refer to the general rebuttal section on binding.

(B) "Binding between MEC modules" - what is specifically meant ...

To clarify, in our paper: binding is just another name for conjunctive composition (please see line 67).

In our model, the state of each grid module $\mathbf{\hat{g}}\_i$ depends on binding the states of other grid modules ($\mathbf{\hat{g}}_{j \neq i}$) and hippocampus ($\mathbf{p}$) (per Eqs. 6 and 7). Our claim in Line 319 is that sigma-pi neurons in MEC implement this binding.

Because if we only mean ... then sure, that's trivial ... If you mean some specific mechanism ... then what is it?

We mean multiplication, implemented by sigma-pi neurons with nonlinearities in dendritic compartments [57].

How does it manifest ... How can we assess that?

We don't know yet, but it's biologically plausible, since nonlinear computations on EC inputs within HC dendrites are important for assigning contexts to place cells [26]. It remains to test if a) these computations implement binding, and b) similar operations occur within MEC grid modules.

(C) ... are they properties of the brain?

To test this, there are methods (structural and functional imaging) for measuring the timescale and persistence of synaptic plasticity.

[A6] George et al. (2023), eLife

Thank you for accurately summarizing our study and capturing the core strengths of the paper. We appreciate your questions, as we think they get at the fundamental context surrounding the paper. We've done our best to address each of them fully and concisely below.

The model’s complexity might pose challenges for practical implementation and experimental validation in biological systems.

• We provided code, which provides proof of concept for practical implementation and lowers barriers for experimental validation.
• In addition, we envision that the model can be implemented with spiking neural networks, as recent works (e.g., [A4]) have done for complex-valued vectors. Such implementations could also help bridge the gap to practical implementation.

1. How biologically plausible is the RNS as a coding mechanism in the hippocampal-entorhinal circuits?

Prior work [4, 5, A5] suggests that the RNS is not only biologically plausible but indeed realized by the brain. This idea is consistent with the striking organization of grid cells into discrete modules along the dorsoventral axis [9].

Are there any existing biological structures that directly support this model?

Yes. In addition to the partition of multiple grid units into discrete scales [9], there are the complimentary roles of lateral and medial entorhinal cortex in processing spatial and non-spatial relations, respectively) [39], and the role of hippocampus as an index to many possible patterns [40]. Throughout the paper we have also tried to highlight the consistency with biological structures and experiments (e.g., lines 124-6, 241-3, 260-3).

2. What specific experiments could be designed to empirically test the predictions made by the model? How feasible are these experiments with current technology?

Important questions. We address these points in the general rebuttal under suggestions for experimental neuroscience. We believe the experimental methods and analyses outlined there are feasible with current technology.

3. The model suggests encoding contexts as vectors in the entorhinal cortex. How does it manage the vast diversity and complexity of possible contexts in real-world environments?

Good question. Assigning contexts to vectors provides a simple but explicit way of measuring the similarity between two contexts (computing their inner product). Indeed, such vector embeddings are widely adopted in machine learning (e.g., word2vec). The situation we consider explicitly in the paper (e.g., in Appendix C.2, Figure S2) are discrete contexts, in which no contexts are similar, since it mirrors global remapping in the hippocampus.

We should caveat that there is an extensive literature on contextual remapping in the hippocampal formation, and we do not attempt to explain every finding in detail. However, an interesting direction for future work is to consider contexts that have manifold structure (for example, in which environmental boundaries have continuously varying color) and applying our model to such cases.

4. While the model shows robustness to noise in simulations, how would it perform under the more complex and varied types of noise encountered in biological systems?

To more fully answer this question, we have conducted some additional experiments under biologically motivated kinds of noise: synaptic failure (dropout), limited synaptic precision (bounded synapses), and ablations/lesions (cell death). The results are summarized in the PDF attached to the general rebuttal in Figures X1, X2, and X3, respectively. We hope that they demonstrate how the model would fare in the face of "resource constraints" and other disturbances in the weights.

[A4] Orchard & Jarvis (2023), ICON Proceedings

[A5] Stemmler et al. (2015), Science Advances