A multi-region brain model to elucidate the role of hippocampus in spatially embedded decision tasks

Thread [1/3]

Thank you very much for your detailed, insightful feedback and the mention of many related valuable works. We highly appreciate your time helping us to improve the quality of our work. Here, we address your feedback and questions in order, in addition to our modification in the manuscript following your feedback, such as in the Discussion and interpretation of results, among others, all in blue.

Based on the results, the authors propose that place cells that conjunctively integrate position and sensory information was important to solve the task.

This is most likely a typo in the summary. But in case of any misunderstanding, we’d like to clarify that we predicted grid cells (not place cells, which are already experimentally established) that conjunctively integrate position and sensory information, along with an activated non grid EC-HPC pathway, were important to solve the task and replicate experimental findings of conjunctive tuning in place cells.

Regarding weaknesses:

The authors only demonstrated that their model replicated the representations observed in 1 experiment (Nieh et al. 2021). The authors should consider recapitulating 1 other neural phenomena to increase the generality of their proposed model e.g. representational drift (Qin et al. 2023 Nature Neuro.), or increase the variety of simulations e.g. 5 arm decision navigation (Baraduc, Duhamel, Wirth, 2019 Science).

We believe the current scope of our work is well contained. Our work studies multi-region interactions underlying spatial navigation given recent findings indicate the same circuit has a potential role in spatial decision-making (Nieh et al., 2021). The findings of the work is binded to cognitive map theory. Though it is conveniently doable to simulate additional tasks that are not directly relevant, we think adding these tasks would deviate away from the main focus of the paper.

Nonetheless, the authors should consider previous models that use different hippocampal representations for navigation learning (Brown & Sharp, 1995; Arleo & Gerstner, 2000; Foster et al. 2000; Zannone et al. 2018; Kumar et al., 2022).

Thank you very much for your insights! We would appreciate more clarification on the first proposal, or, if you were referring to a potential inclusion of CA3 recurrence, we have added a new paragraph to Discussion justifying our current choice of modeling (lines 516-525).

For instance, will an RNN based RL agent with place cells and sensory evidence as input learn to solve the task (Kumar et al. 2022 Cerebral Cortex; Singh et al. 2023 Nat Mach. Int.), similar to M0 but with place cell position inputs? This proposal also suggests a EC-MEC-hippocampus-neocortical-striatum pathway but was not considered.

Regarding the latter, this is a great point, and we actually have results on the variant of M0 that receives both hippocampal position code and sensory, as well as variants to M1-M5 where RNN receives both hippocampal code and sensory. These are not ultimately included because of the scope of the work, as we think these results will bring added confusion without ultimately affecting the current result. Our consideration and justification of only considering p → RNN is discussed in the last paragraph of Section 3.1 lines 202-208. Below we copy the content directly for your convenient reference:

“ Although MEC, HPC, and EC can all interact with the cortex biologically, we model the place cell vector as the ultimate readout to the cortex given it is minimally sufficient for learning the task and testing the counterfactual of how the co-tuning of place cells arises. This framework enables extensive future studies. For example, one can systematically evaluate the computational advantages of different combinations of {MEC, HPC, EC} input(s) to the cortex for their roles in enabling generalization and rapid learning, e.g, sensory inputs from the EC may be especially important in scenarios where animals need to remember decision positions in the dark. ”

A recurrently connected neural network can have readouts as an additional set of nodes and weights, but it is not a given as we can also have an RNN without readouts to learn representations as in Miconi et al. 2017 eLife. I assume there are a total of 3 sets of weights to train in your "policy network": the input to the RNN, the recurrent weights, and the readout weights. Training 3 sets of weights using deep RL allows the model to learn additional representations/features to solve the task, for instance integrating temporal information (Singh et al. 2023 Nature Mach. Int.). Perhaps the EC-HPC network does not capture the temporal dependencies of the Tower Accumulating task, and instead the RNN is the one that is learning to integrate these temporal features to successfully learn a policy. I am starting to think this might be the case.

A clearer way to construct a policy network is to have only one set of weight matrix $W_{ij}$ so that place cell representations or states $s$ directly synapse to the actions $a_j$ following $a_j = \sum_i^N W_{ij} s_i$, similar to having a classification layer in deep networks. This is done by Zannone et al. 2018 Sci. Rep. who uses the Q learning algorithm and Foster et al. 2000 Hippocampus who uses the Actor-Critic algorithm to update $W_{ij}$. Hence, a simpler policy network only has 1 set of weights, that does not need to be trained by back propagation but can be trained by bio. plaus. rules. A model without the RNN but just the policy network will be more convincing to show that the temporal integration of sensory information is done by the EC-HPC network.

I would strongly recommend the authors to evaluate a new model in which the HPC representations feed directly to the policy network, without the RNN, so that $a_{j,t} = \sum_i^N W_{ij} p_{i,t}$. I hope this is clear?

We are keen to hear your thoughts and hope our clarifications have sufficiently addressed the concerns and provided enough reasons for a consideration of raising the score. Please let us know if we could assist with further comments or questions. Thank you again for your time and feedback.

[1] Montague et al., A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci., 1996.

[2] Schultz et al., A neural substrate of prediction and reward. Science, 1997.

[3] Lee et al., A feature-specific prediction error model explains dopaminergic heterogeneity. Nature Neuroscience, 2024.

[4] Miller et al., Cognitive Model Discovery via Disentangled RNNs. NeurIPS, 2023.

[5] Pinto et al., Task-dependent changes in the large-scale dynamics and necessity of cortical regions. Neuron, 2020.

[6] Bondy et al., Coordinated cross-brain activity during accumulation of sensory evidence and decision commitment, 2024

[7] Brunton et al., Rats and humans can optimally accumulate evidence for decision-making. Science, 2013.

[8] Gershman & Ölveczky, The neurobiology of deep reinforcement learning. Current Biology, 2020.

I would like to thank the authors for clarifying some concerns, especially with the analysis. However, I am inclined to maintain my score as I feel the model is unnecessarily complex and the analysis is only performed in one simulation environment.

Unlike the TEM model (Whittington et al. 2020), the Vector Hash model has grid and place codes using bio. plaus. rules which is nice. Furthermore, it has been shown that training just the readout layer of a feedforward/RNN using the temporal difference Hebbian learning algorithm allows agents to learn complex navigation policies (Kumar et al. 2022 Cerebral Cortex). The authors should have gone with a purely biologically plausible navigation model since the main computation is performed by Vector Hash, and the RL signal is not backpropgated to learn the representations for navigation like in deep RL. This will make the model much more interpretable and easier to train for others to replicate, given the difficulty the authors faced as mentioned in Appendix A.1. The authors should discuss why they chose the RNN + policy network and not simpler alternatives. Will this change the conclusion of the model?

Lastly, the authors should have shown the generality of the model in a variety of 1D and 2D environments instead just one environment.

I appreciate the authors' clarifications. The model's strength lies in its rapid map learning, akin to episodic memory. However, I am concerned about its ability to adapt to other neuroscience experiments.

Additionally, our use of an RNN abstracts the subcortical and cortical regions (lines 62-63), which are integral to the decision-making process. [Author Response]

The authors mentioned that M0: RNN + RL network is not able to solve the task (Fig. 2), and the grid+place cell network added to the RNN improves learning. Hence, it seems contradictory to me that the authors emphasize on the importance of using an RNN to model decision-making again. As mentioned before, it will be much clearer to show that a grid-place-policy network, without the RNN, improves policy learning. Please see the list of references given before that shows a simpler model and grounding to neuroanatomy, which will make it easier to adapt the current model to other simulations. To me, overcomplicating a model to show multi-region interaction is obscuring the computational understanding offered by simpler alternatives.

Such an understanding could reveal how cognitive maps could be leveraged not only to navigate physical spaces but also to guide cognitive decisions. [Pg. 1, Line 51]

We believe the current scope of our work is well contained. [Author Response]

I would like to re-emphasize that the current scope of the experiments to make the claim that the 'cognitive' map learned by the model improves policy learning is insufficient. The authors should have shown at least 1 additional simulation in a different environment perhaps with obstacles, or a non-spatial but cognitive task to further validate that the representations learned by the model is general.

Another suggestion is to show that the model replicates the rapid learning seen in Tse et al. 2007 Science by learning schemas. Some modeling papers include Hwu & Krichmar 2020 Biological cybernetics and Kumar et al. 2021 arXiv 2106.03580.

The current model is capable of improving our understanding on a range of interesting cognitive questions e.g. schemas, beyond the tower accumulation task. I urge the authors to consider these suggestions to strengthen the model's robustness.

Thread [1/3]

Thank you very much for your detailed and thoughtful feedback! We highly appreciate your time helping us to improve the quality of our work. Here we address your comments in order, in addition to modifications made to the manuscript, including Fig 1C, 1E, Discussion, and other minor edits in blue for added clarity on methodology and result interpretation.

Regarding weakness:

For instance, CA3 in the hippocampus has extensive recurrent connections…This simplification may have critical effects when studying how the model produces co-tuned place cells and choice-specific firing since it removes internal computation within the hippocampus, relying instead on upstream grid cell networks.

The consideration of CA3 recurrence is a very valid point, but we didn’t include it due to the scope of our work. We have added an additional paragraph discussing this limitation to our Discussion section. For TLDR; reasons are several-fold, such as the reasons being brought up by reviewer bpUy (Question 1) that we focus on entorhinal-hippocampal interactions due to its importance to cognitive map theory and spatial navigation. Furthermore, considering CA3 recurrence is, in fact, complementary to our work, as testing the counterfactuals of CA3 recurrence would still involve M0-M5, and their variants. This extra consideration can be easily tested using our framework as detailed in the additional paragraph in Discussion, but it adds an extra layer of complexity, making it outside the scope of this work. Since our work generates a straightforward, falsifiable prediction, we are currently collecting neurophysiological data to verify our predictions–the result will directly inform whether other mechanisms play a role, such as CA3 recurrence, and can be easily investigated in our current framework as described in lines 518-522.

For a balanced evaluation, it would be essential to compare the performance of a standalone RNN (without Vector-Hash) receiving both physical velocity and sensory evidence, with an equivalent number of neurons.

Thank you for pointing this out. M0 is a standalone RNN, intaking sensory only. All model variants have RNNs of the same size (hidden size of 32, indicated in Appendix A.1). The point of including M0 is to provide a baseline when an abstractor of velocity information, i.e., VectorHaSH, is absent. If the standalone RNN instead receives abstracted velocity information, in contrast to other models that receive hippocampal representations, the task would be trivial. We could be misunderstanding the proposal, e.g., did you mean to have an RNN of size 32 to receive sensory vectors concatenated with MLP-abstracted evidence velocity, instead of just the sensory vectors in the current setup? Currently, in M0-M5, no RNN receives velocity information directly. Any clarification would be appreciated, thank you!

Thank you for your response and for taking the time to update the manuscript. However, after reviewing your reply, I feel that my primary concerns regarding the weaknesses of the study have not been fully addressed. I would like to elaborate further, especially considering the importance of these points to the validity of your core conclusions.

1. On the exclusion of CA3 recurrence:

One of the key contributions of your paper is the prediction that joint integration of velocity and evidence in grid cells is critical for producing co-tuning of place cells in the hippocampus. However, as I mentioned in my review, this conclusion may be significantly influenced by the structural simplifications in your model, particularly the omission of CA3 recurrent connections. The hippocampus, as a central component of the cognitive map, is widely understood to integrate information from multiple sources, and the recurrent connections in CA3 are thought to play an essential role in this integration. While I understand that the scope of your work focuses on entorhinal-hippocampal interactions, simply stating that your prediction is falsifiable does not justify omitting CA3 recurrence from your analysis. Instead, it is essential to explain why the exclusion of CA3 recurrence does not compromise your prediction or, at the very least, discuss how it might influence the interpretation of your results. This is especially crucial given that your model already incorporates projections from the lateral entorhinal cortex (LEC) to the hippocampus, which should facilitate evidence integration in the hippocampus.

Considering the central role of this prediction in your study, a more thorough justification or analysis is necessary to demonstrate that the lack of CA3 recurrence does not undermine your conclusions. Otherwise, the validity of this prediction remains questionable and risks being an artifact of the model's structural oversimplification.

2. On the role of a standalone RNN:

In your manuscript, the first section of the Results is titled "Joint Integration of Position and Evidence in MEC Induces Rapid Learning". This title implies that introducing joint coding of velocity and evidence in the Vector-Hash model (e.g., M3 and M5) leads to faster learning compared to a standalone RNN (e.g., M0). Indeed, Figure 2A shows that M3 and M5 achieve higher accuracy more quickly than M0. However, I believe this comparison is not entirely fair for the following reasons:

• The RNN in M0 has significantly fewer total neurons than the combined RNN and Vector-Hash modules in M3 and M5, which inherently puts M0 at a disadvantage in learning capacity.

• The RNN in M0 receives only sensory information as input, whereas the Vector-Hash models also encode velocity information, which is highly task-relevant.

To make a fair comparison, I suggest including an additional condition where an RNN with the same number of neurons as M3 or M5 receives both velocity and sensory inputs. This would allow readers to directly compare the learning capabilities of the standalone RNN and the proposed RNN + Vector-Hash model.

If your intention is not to claim that the RNN + Vector-Hash model learns faster than a standalone RNN, then the phrase "induces rapid learning" should be clarified in the manuscript. Specifically, the comparison target for this claim should be explicitly stated to avoid potential misinterpretations.

Thank you very much for the insightful and timely elaborations! We now better understand the concerns, and have conducted relevant experiments.

In addressing concern (1) of CA3 recurrence, we have added Fig 11 and Fig 12 to Appendix F. These results serve as a proof of concept, that adding CA3 recurrence to M2 (position only grid code + mix p) or M4 (disjoint grid code + mix p) is not sufficient to drive experimentally observed phenomena in place cells, showing such a model simplification does not undermine our conclusions drawn in the main text. We have modified the relevant text in Discussion accordingly, highlighted in orange in lines 517-525, elaborating the implication and future directions.

In addressing concern (2) of baseline comparison, we have included an additional standalone RNN baseline to Fig 2 in black. We additionally scaled up the original standalone RNN baseline in blue. Specifically, the standalone RNNs now have a hidden size of 32 + Ng + Np + Ns (same number of neurons as RNN + Vector-HaSH in M5). The RNN takes in sensory info, while for the black variant, it is concatenated with position velocity and evidence velocity (predicted by MLP). The previous conclusion remains the same, except we observe learning/training instability in these large-size RNNs--the implementation details are elaborated in Appendix A.1.

Thank you again for your time and valuable feedback! Please let us know if further clarifications are needed in addressing your concerns. We hope that our response and corresponding edits provide sufficient reasons to raise the score.

Thread [2/2]

Regarding ML relevance:

How can these findings be more directly linked to representation learning and machine learning at large?

Neural representations, such as those of place cells and grid cells, are emerging properties of learning within biological neural networks; these representations can arise in artificial neural networks with appropriate biologically-imposed constraints as shown in our work. We study these representations using ML frameworks (lines 131-139). Our study on understanding how distinct cognitive processes like decision-making and spatial navigation are jointly possible in a limited number of interacting neural networks (brain regions) is of interest to both ML and Neuroscience communities. Navigation is very difficult for AI (Mirowski et al., ICLR 2017), and it’s even more so when they are combined with decision-making. Our understanding of what representations are needed to efficiently solve the combination using a limited network circuitry, with verifiability in both biological and artificial agents, is an important contribution. Further, our study matters for figuring out how artificial agents can truly make sense of their environments while being energy efficient, capable, and cognitively flexible, like humans and animals. This is related to achieving autonomous machine intelligence positioned and discussed by LeCun, 2022; we demonstrated, as a proof of concept, that sample-efficient learning in RL is achievable through an external content-addressable associative memory with a structured aspect. In particular, our work shows a structured conjunctive coding scheme (i.e., grid cells as a canonical example drawn from biological representation learning) is an important structural representation for forming cognitive maps (world models), enabling individuals to learn quickly and navigate spaces efficiently.

We have revised our introduction with the above to emphasize the relevance of our work to ML (lines 65-70, in blue). Additionally, our work showcases ML techniques applied to neuroscience discoveries (our primary area of submission) for the reasons stated when addressing weaknesses. Thank you again for your feedback!

Section 2.3 is an odd aside, and the text does not seem to link it to the proposed model or findings at hand.

We kindly disagree with the reviewer’s comment that Section 2.3 is an unrelated aside. The ML relevance of grid cell representation, which is the main subject of our study, is directly backed by the literature we mentioned in Section 2.3, e.g. Banino et al. demonstrated that artificial agents with grid cell-like representations have superior performance in navigation, a difficult ML task. Our findings provide additional insights, e.g., grid cell-like coding should also be efficient, i.e., utilizing all axes of the representation space (”conjunctive tuning”) in each module.

Regarding ethics concerns

Fig 1A is adapted from Chandra et al., 2023 with proper citation (line 179). Fig 1B is created by us, not from Chandra et al., 2023.

---

Nieh et al., Geometry of abstract learned knowledge in the hippocampus. Nature 2021

LeCun, Yann. "A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27." Open Review 2022

Banino et al., Vector-based navigation using grid-like representations in artificial agents, Nature 2018

Mirowski et al., Learning to Navigation complex environments, ICLR 2017

---

Please let us know if further clarifications are needed. Thank you again for your time and valuable feedback! We hope that our response provides sufficient reasons to raise the score.

Thread [1/2]

Thank you very much for your valuable time and feedback in helping us improve our work! Here we address the weaknesses and questions respectively, in addition to the revision made in Introduction to emphasize on the ML relevance of our work (lines 65-70, in blue)

Regarding weakness

Overall, it is difficult to follow the logic of the paper. The introduction begins with a very specific example of neurophysiological findings and then seeks to justify the choice of Vector-HaSH.

We would like to respectfully point out that the neurophysiological findings we reference (Nieh et al., 2021, Nature) are not just specific examples but are representative of broader and well-established principles in cognitive map theory, which is foundational for our study (Section 2.1). We chose these findings because they offer robust experimental evidence regarding place cell conjunctive tuning of physical and cognitive variables, which serves as a strong ground truth for testing hypotheses in our model. By aligning our simulations with these generalizable experimental results (Fig 3, 4), we demonstrate the biological relevance and predictive power of our ML-oriented approach. Therefore, the introduction of these findings is not only logical but essential for grounding the context of our work.

the proposed model is a slight modification of this previously published work, by the introduction of an MLP

Relatedly, the model is a small modification of the previously published vector-hash...

We would like to kindly point out that our method is not just a simple modification of Chandra et al., (2023). While it is true that our approach to modeling the entorhinal-hippocampal interactions is based on Vector-HaSH, it's not the entirety of our multi-region model (Section 3.1, Fig 1B, D, E; e.g., we introduce a new framework for modeling the entorhinal-hippocampal-neocortical loop, as highlighted by Reviewer LP6J). The model is established to be biologically grounded (concise reasons listed in lines 121-122), serving our purpose of building a neural computation testbed given its mechanistic nature and proper level of abstraction; other well-constructed mechanistic models of entorhinal-hippocampal interactions are not available to our best knowledge. Thus, we did not reinvent the wheel. However, the model’s

1. interaction with cortical and subcortical regions,
2. goal-directed task performance and neural representation under different neural computational rules,
3. application to neuroscience discoveries as an ML model,

are unexplored—we explored all the above. The main contribution of our work is a biologically plausible ML-based framework (lines 77-82) that serves as a testbed of hypothesis-driven neural computation discoveries (Table 1, 2, Section 4). Further, we directly leveraged this framework to make straightforward, falsifiable neurophysiological predictions in important topics of neuroscience (Section 2.1). This aspect alone already makes our work well suited and important in our primary area, “applications to neuroscience & cognitive science.”

Thread [1/1]

Thank you very much for your time, and insightful, detailed comments. We truly appreciate your kind words, and your valuable time helping us to improve the quality of our work. Here we address your questions and feedback in order, with modifications applied to Discussion, Fig 1, Fig 5 caption; we added Fig 10, and additional text to Method (in blue):

…is it possible that the conjunctive representation in HPC is also due to internal reasons e.g., interactions between HPC subregions? The authors might want to add a sentence or two to highlight why they chose to study these two reasons specifically.

Thank you–it is a very valid point regarding the internal interactions within HPC subregions. Aside from the reasons you mentioned, we did not include it for reasons we just added to the Discussion section of the manuscript (in blue, lines 516-525). TLDR; being the consideration of e.g., CA3 recurrence would be complementary to our current work. And, given we have a falsifiable and straightforward prediction of conjunctive grid code, we are collecting neurophysiological data to verify this prediction directly. The result will directly inform whether other mechanisms play a role, and can be easily investigated with our current framework with methods described in lines 518-522.

However, it is unclear how position is represented and input into the grid cells. Is it encapsulated in the CAN module?

Positional velocity is represented as 0 (stuck) or +1 (forward) without the use of MLP given backward-moving is not task-relevant (Nieh et al., 2021, see behavioral training), though it is possible to use an MLP as well. It will not affect the results. Generally, the velocity inputs to each grid module updates the grid phases through path integration, following Vector-HaSH implementation in Chandra et al., 2023, akin to Burak & Fiete, 2009. For added clarity, we added a grid code schematic of how velocity is represented with respect to inputs in Fig 1C and elaborated in lines 198-200 and Appendix A.1.

The MLP module models how evidence is extracted from sensory inputs, and from Fig1B, it seems to receive sensory inputs directly from non-grid EC…what neural mechanism does this MLP correspond to? Maybe something to include in the Discussion.

In our model, the sensory input and the encoding of non-grid EC are technically the same as a consequence of our setup (lines 215-217). The MLP predicts velocity from sensory directly instead of from non-grid EC (lines 200-201). We agree our schematic was misleading, and thank you a lot for pointing this out! We have modified Fig 1B and 1E correspondingly. We have also added to the Discussion our justification of using an MLP (lines 526-532 in blue), in addition to the earlier mention of its potential biological implication in lines 227-228.

How did you achieve disjoint grid cell encoding of space and evidence in your model? i.e., computationally and mathematically, how did you make a difference between joint and disjoint grid cell codes? You may want to include this in section 3.2 Model Setup, or in the appendix.

Thanks for pointing this out! We added Fig 1C for added clarity and elaborated in lines 198-200 and Appendix A.1 (as mentioned above in pt. 2). The joint/disjoint grid coding scheme is a manual setup so we can have a careful, controlled comparison of both possibilities. Please let us know if it’s still unclear, and we are happy to clarify more.

In line 309 'we show that our multi-region brain model (M4)' - you mean M5 right?

Yes, thanks. This is now fixed.

Fig3 left: if space allows, you may want to include Fig1d from Nieh et al. 2021 to make a contrast to right-choice-selective place cells, as readers unfamiliar with their work might struggle to understand what this diagram means.

Yes, it was a part of the consideration, but we removed it since Nieh et al. confirm that Fig1e was the experimental outcome, so we didn’t want to confuse readers with the other possibility that was ruled out experimentally.

Section 5.3.1: I feel this whole subsection is a bit unclear. I wish you can check/clarify the following points…

Thank you for pointing this out and for your constructive feedback! We have made modifications with respect to each point. Specifically, we modified the section 5.3.1 title and fixed the relevant typo. We have included the separability of accumulated evidence in Figure 10. We have also reworded Fig 5 caption.

---

Please let us know if further clarifications are needed. Thank you again for your time and valuable feedback!

Thread [2/2]

Could the author provide a mechanistic insight and some supporting analyses?

Intuitively, grid code provides rigid information of an environment and task at hand, but non-grid input to HPC is potentially helpful to capture the nuances in the environment, such as where the wall is (hinting the decision region). For added clarity, we added lines 357-359 in blue. Thank you for your feedback!

The mechanistic insight of M5’s faster learning can be supported by the analysis in Fig 4, A3, showing splitter cell phenomenon, in which place cells show differential activity based not only on spatial location but also on additional context (”choice-specific”). This is also supported by analyses of Fig 5, B, and quantitative measure in the new Fig 10, showing the hippocampal representation is (visually) separable in low dimension under joint grid code and activated EC pathway.

The comparison between model dynamics and biological data appear largely qualitative…how can we be sure there are no separable clusters in high dimensional space?

We focus on low-dimensional representations, especially given the task is low-dimensional and Nieh et al. found the experimental hippocampal population activity is constrained to a low-dimensional manifold. High dimensional separability is very likely, but even so, given M5 is separable in low dimension while other variants are not, this characteristic alone underscores the computational advantage of mechanisms implied by M5. We have modified the wording in section title of 5.3.1 to reflect our intention. We also appreciate your feedback on more quantitative measures, and have added scree plots of M1-M5, i.e., cumulative variance explained by numbers of PCs, to the Appendix Fig 10 (row 1). Quantitatively, M5's first 2 PCs of hippocampal activity can explain 68% of variance, while M3 only explains <20%, similar to other models. Though the comparison of place cell coding is qualitative-oriented in Fig 4, the place cells are selected through a quantitative mutual information analysis (Appendix B).

Other implementation details, such as how the RNN is trained in an RL framework, is lacking.

The implementation details are mentioned: the RL training was briefly described in line 248, “which is an action-selection RNN policy trained through policy gradient under reinforcement learning…” which points to a more detailed description in the Appendix A (line 249), “Please refer to Appendix A.3 for what one step by the agent in the environment entails among the involved brain regions.” These are also mentioned in line 312.

---

Please let us know if further clarifications are needed. Thank you again for your time and valuable feedback! We hope that our response and corresponding edits provide sufficient reasons to raise the score.

Thread [1/2]

Thank you very much for your valuable time and feedback in helping us improve our work! Here we address the weaknesses and questions. Additionally, to enhance the clarity of our methodology, we have added additional figures, Fig 1C, 1E, Fig 10, and further elaboration in lines 198-200 and Appendix A.1.

The neocortex is modeled as a MLP and appears to the only channel by which sensory input enters the EC-HPC circuit.

We’d like to kindly clarify—sensory information in non-grid EC layer (Fig 1B green) can also enter the circuit (M2, M4, M5; detailed in Table 1, 2).

Since the MLP module plays the important role of extracting the evidence velocity and conveying to the Vector-HaSH module, it is unclear comparison with Models 0-2, which do not have the MLP module, is fair.

We believe the comparison is rigorous and logical. The main goal of the paper is testing hypotheses of neural computation, as shown in Table 1–the point of including M1, M2 is to rigorously span the entire hypothesis space of how conjunctive coding in place cells arise. To put Table 2 more clearly, the inclusion of M1-2 follows a simple logic, where the hypothesis space is

The point of including M0 is to provide a baseline when an external integrator of velocity information, i.e., VectorHaSH, is absent.

It is not entirely clear how the different coding…joint coding scheme emerged in M4…Do different grid codes in Models 3-5 emerge due to difference in the architecture? Or other network parameters?

Thank you for pointing this out! Line 309 (now line 304) was a typo, and is now fixed. The grid coding scheme is carefully controlled by us to test how different coding schemes affect the downstream computation, e.g., place cell representation, task performance.

To add clarity, we added Fig 1C and 1E in the latest version to illustrate the differences between the coding scheme and model architecture, and elaborated in lines 198-200 and Appendix A.1. As Fig 1C shows, the disjoint grid code means each grid module only gets velocity input of one variable, represented by one axis of the 2D representation space; the joint grid code means both axes of the 2D space are utilized in each grid module. Both coding schemes are capable of providing distinct code for different states of position and evidence, but the downstream representation and performance turns out to be different.

The above also addressed both questions you had. The hyperparameters such as network size and learning rate are shared across models–these details are stated in Appendix A.1. Please let us know if anything else is still unclear, and we are more than happy to elaborate.