Grid Cell-Inspired Fragmentation and Recall for Efficient Map Building

Thank you for your insightful and constructive feedback on our manuscript. We greatly appreciate the time and effort you have dedicated to reviewing our work. We will revise the manuscript, taking into account each of your valuable comments. We would like to emphasize that the core contribution of this research is establishing a novel connection between neuroscience and SLAM, rather than proposing a state-of-the-art SLAM algorithm.

Re: the improvement over the alternative methods seems to shrink in the more real-world-like applications. Can the authors comment on or provide an interpretation of this?

In robot simulations, there is significant noise from the LIDAR sensor, path planning, actuation, and localization of the current position. These factors lead to increased memory usage in both the Frontier and FARMap methods. Consequently, the performance gap between these two methods is smaller than in the proposed environments. Additionally, we measured the memory usage of FARMap based on its peak usage during navigation. While the maximum usage of FARMap is similar to its counterpart in Environment 1 and in American settings, the overall usage is consistently lower. For instance, in AWS, FARMap explores 49% more of the environment while using only 37% more memory.

In robot simulation, there are many noise from LIDAR sensor, path planning, actuating, and localizing current position. They lead to use more memory on both Frontier and FARMap. Therefore, the gap between two methods is smaller than the proposed environments. On the other hand, we measure the memory usage of FARMap as the largest memory usage in navigation. Although the maximum usage of FARMap is similar to its counterpart in Environment 1, and American, the overall usage is smaller. In AWS, FARMap explores 49% more environment while it uses 37% more memory usage.

Re: Can the authors justify why the Frontier method by Yamauchi (1997) would be the most appropriate benchmark to have?

We selected the Frontier method (Yamauchi, 1997) as our baseline because it is a fundamental technique widely used in contemporary methods (Kulkarni et al., 2022) and is often employed as a benchmark (Chaplot et al., 2020). Its widespread adoption positions the Frontier method as an ideal candidate for integration with FARMap. This integration facilitates the possibility of employing various methods for local subgoal generation, thereby enhancing the versatility and applicability of FARMap.

Re: I am not sure if “grid cell-inspired” in the title is entirely appropriate given that the authors do not use grid cells in their model and the idea of representing space in segments seems to hold more generally for the hippocampal-parahipponcampal representation in rodents.

As the reviewer pointed out, the concept of remapping is indeed associated with place cells in the hippocampus, not exclusively with grid cells (as noted in the second paragraph of our Introduction). However, our approach employs surprisal changes as a criterion for fragmentation, which correlates with the locations of actual grid cell remapping, as shown in existing studies (Klukas et al., 2021). Therefore, we titled our paper “grid cell-inspired”. We will change the paper title reflecting on your comment if the paper is accepted.

Thank you for your insightful and constructive feedback on our manuscript. We greatly appreciate the time and effort you have dedicated to reviewing our work. We will revise the manuscript, taking into account each of your valuable comments. We would like to emphasize that the core contribution of this research is establishing a novel connection between neuroscience and SLAM, rather than proposing a state-of-the-art SLAM algorithm.

Re: Some of the general ideas are already present in other methods for exploration. The paper should discuss the proposed method with respect to such methods.

We appreciate the reviewer's acknowledgment of our discussion on graph-based SLAM in Appendix A. The topological graph in FARMap and the Generalized Voronoi Graph (GVG)-based approaches are designed for different objectives. GVG-based methods primarily aim for rapid path planning through pre-computed paths and loop closure. In contrast, FARMap's topological graph is designed to surmount the limitations of local maps that cannot refer to other submaps. However, this does not mean that both methods cannot be used together. Rather, there is a potential synergy to use both as we mentioned in Appendix A.1.

Regarding curiosity-driven exploration [1,2], these methods employ future prediction error or surprisal as intrinsic rewards to address environments with sparse rewards. Conversely, FARMap utilizes surprisal for segmenting space in accordance with areas where grid cell remapping occurs.

[1] Pathak, Deepak, et al. "Curiosity-driven exploration by self-supervised prediction." ICML, 2017.

[2] Burda, Yuri, et al. "Exploration by random network distillation." ICLR, 2019.

Re: the paper's comparison is limited in considering only the standard frontier-based exploration.

Thank you for this suggestion. FARMap is designed to be versatile and can integrate with existing exploration methods. Exploration-RRT which is a kind of multi-goal RRT can work as a subgoal generator and path planner in FARMap. Additionally, Caley et al. and Shrestha et al. predict goal location from predicted map structure of indoor environment so it is easily adapted as a local subgoal generator. On the other hand, TARE divides space outside of current local planning horizon into multiple cuboid subspace and each subspace is labeled as explored, exploring and unexplored. In this case, FARMap can be combined with two ways: 1) employing only local path planning for the exploration strategy within local maps, and 2) combining both local and global path planning strategies in local maps, using larger fragments.

We also wish to highlight that we have conducted comparisons with learning-based exploration methods, such as Neural SLAM, in Section 5.3, and with curiosity-driven exploration methods, like RND, in Appendix J.

Re: for the realistic experiments, in AWS office, memory appears better for Frontier.

We acknowledge that in the AWS office scenario, FARMap demonstrates higher memory usage compared to Frontier, contrary to the results in our proposed environments where FARMap and Frontier have a significant gap in memory usage but achieve better map coverage. However, it's important to consider that coverage directly impacts memory usage. In AWS, FARMap explores 49% more of the environment while using only 37% more memory, showcasing its efficiency in broader exploration.

Re: The size of each local map might depend on the complexity of the environment, but it is worth discussing what affects the determination of the local map in practice.

The size of local maps in FARMap is influenced by two key factors: the environmental complexity and the fragmentation threshold, $\rho$. In more complex environments, surprisal tends to be higher compared to simpler environments, as the complexity often limits the field of view and increases the time required to move from one location to another. However, generating an excessive number of local maps can lead to unnecessary overhead in communication between Short-Term Memory (STM) and Long-Term Memory (LTM), and may also result in each map covering too small a region. Conversely, without any fragmentation, the benefits of using FARMap are diminished. To strike a balance, we employ the z-score of surprisal to regulate the number of fragments – minimizing excessive fragmentation in complex environments while ensuring sufficient fragmentation in simpler ones. Then, $\rho$ modulates the amount of fragments.

We appreciate the rapid response of the reviewer. We are sincerely pleased that the reviewer acknowledges the connection. We will strengthen this aspect in the revised manuscript.

Re: One element that is worth elaborating in the paper then is also discussing the hyperparameters, such as decaying factor and fragmentation threshold, based on those studies and see whether they align with the values set

Thank you for the pointing out. Unfortunately, both Derdikman et al. (2009) and Carpenter et al. (2015) do not deal with decaying factor and fragmentation threshold since their focus is to empirically show that the fragmentation (remapping) happens in a continuous environment, and how grid cell representation looks like or changes. To the best of our knowledge, there is no neuroscience work that mentions both decaying factors or fragmentation criteria along with actual empirical studies. Rather there is a separate study for decaying theory in psychology that we can refer to the factor (please refer to the review paper from Ricker et al.).

Ricker, Timothy J., Evie Vergauwe, and Nelson Cowan. "Decay theory of immediate memory: From Brown (1958) to today (2014)." Quarterly Journal of Experimental Psychology 69.10 (2016): 1969-1995.

Re: The non significant changes in performance might also come from the scale of those hyperparameters: while the analysis included values for example from 1 to 3 for the fragmentation threshold, practical values that can be derived from those studies will have a larger range.

We would like to remind that the fragmentation threshold is intended to reflect the complexity of the local map using a $z$-score. The range of 1 to 3 was chosen based on the statistical rule of thumb; the probabilities of a $z$-score exceeding 1 and 3 are 16% and 0.15%, respectively, in a Gaussian distribution. Note that we defined the fracture point as a high surprisal point. However, since the surprisal is a non-i.i.d sample, the distribution might not be Gaussian. We plan to conduct a broader sensitivity analysis, although it cannot be added during the remaining rebuttal period.

However, for Table 8, we conducted nine different fragmentation threshold tests directly using surprsial instead of using $z$-scores in large environments. Note that the range of surprisal is in [0, 1]. Table R1 shows that performance varies depending on the threshold; low thresholds lead to excessive fragmentation, while a very high threshold (0.9) reduces fragmentation but increases planning time. This demonstrates the challenge of defining the correct fragmentation threshold for each environment, as opposed to using a statistical approach (i.e., $z$-score).

Table R1. Average coverage, memory usage, and time of FARMap with various fragmentation thresholds without using $z$-score in a large environment.

Thank you for your insightful and constructive feedback on our manuscript. We greatly appreciate the time and effort you have dedicated to reviewing our work. We will revise the manuscript, taking into account each of your valuable comments. We would like to emphasize that the core contribution of this research is establishing a novel connection between neuroscience and SLAM, rather than proposing a state-of-the-art SLAM algorithm.

Re: What does the map representation look like?

We apologize for the confusion regarding the local map and surprisal. The local predictive spatial map is a $C+1 \times H \times W$ dimensional tensor. In each coordinate (h, w), it represents C+1 dimensional vectors, where the first C dimensions (0 to C-1) denote color and the last dimension denotes confidence. If the observation is RGB as in the proposed environments, C is 3. Similarly, C is 1 if the observation is in grayscale. On the other hand, if the observation comes from an occupancy sensor (i.e., LIDAR in Section 5.2 and Neural SLAM module in Section 5.3), C is 0, which means that there is no color. We will clarify this in the revision.

The value of the C-th dimension represents the confidence of each cell observed during exploration. It is gradually decaying like a biological agent’s working memory as we mentioned in Section 3.1. We kindly remind you that the primary goal of this paper is to apply neuroscience theory and foundations to SLAM.

Please refer to the attached codebase (lib/memories/memory.py) for the detailed actual implementation. If you have further questions, please let us know. We will be happy to answer them.

Re: Equation 2 is also highly confusing

The C-th channel of the local predictive map, $\mathbf{M}^\text{cur}_{t-1},C \in \mathbb{R}^{H\times W}$, where H and W represent the height and width of the map, contains the confidence level of each cell. If a cell has been observed before, it holds a nonzero value; otherwise, its value is 0. On the other hand, the C-th channel of the spatially transformed observation at time $t$, denoted as $o’_t,C \in \mathbb{R}^{H\times W}$ (refer to Figure 3 for a visual representation), indicates whether a cell is currently visible (1) or not (0).

Therefore, $\mathbf{M}^\text{cur}_{t-1}, C \cdot o’_t,C$ represents the summation of the confidence of each visible cell in the local predictive map before updating the current (time $t$) observation. Similarly, $|| o’_t,C ||_1$ represents the number of visible cells in the current observation. Hence, Eq. (2) denotes the average confidence of a visible cell from the current observation.

Re: “Surprisal - How can an error be used as uncertainty estimation?”

Surprisal is usually defined as future prediction error or distribution difference in many research areas such as psychology [1], reinforcement learning [2], and neuroscience [3]. However, surprisal can also be interpreted as uncertainty (predicted variance), as demonstrated by Kendal et al. [4]. Therefore, we refer to it as ‘uncertainty estimation’ in our work. We will revise the terminology for clarity in our revised manuscript.

[1] Modirshanechi, Alireza, Johanni Brea, and Wulfram Gerstner. "A taxonomy of surprise definitions." Journal of Mathematical Psychology 110 (2022): 102712.

[2] Achiam, Joshua, and Shankar Sastry. "Surprise-based intrinsic motivation for deep reinforcement learning." arXiv preprint arXiv:1703.01732 (2017).

[3] Sinclair, Alyssa H., et al. "Prediction errors disrupt hippocampal representations and update episodic memories." PNAS 118.51 (2021): e2117625118.

[4] Kendall, Alex, Yarin Gal, and Roberto Cipolla. "Multi-task learning using uncertainty to weigh losses for scene geometry and semantics." CVPR. 2018.

Re: Does the method assume perfect localization and if so how does it handle realistic uncertainty in pose?

We do not assume perfect localization. Results from Robot simulations using a LIDAR sensor (Section 5.2) and Habitat simulations with the Neural SLAM module, which predicts 2D observations from RGB egocentric observations (Section 5.3), demonstrate that our method works well in noisy, continuous environments without assuming perfect localization.

Re: Table 2 shows memory usage with a unit of (k), what does this mean?

In Table 2, 'k' is used to denote 1000, representing the actual discovered size of the environment (in cell units with the resolution mentioned in Appendix D.2). Regarding resolution, we use the default resolution of turtlebot, 0.05 m x 0.05 m. We will modify the notation to (x1000) in future revisions and convert these values into square meters (m^2) for clarity.

Re: The paper provides statistical information which is good, though it might be better if either the standard deviation or quantile (likely the better choice) were used throughout rather than switching between the two.

Thank you for your comment. We employed a 95% confidence interval from bootstrapping for the proposed environments, as the results are aggregated from various maps not from the same map (distribution), making standard deviation inappropriate. Conversely, in the robot simulation in ROS, we can calculate a 95% confidence interval based on standard deviation as it was run multiple times in the same environments. We will adjust the notation of the 95% confidence interval in Table 2 accordingly.

Re: Is the baseline method utilizing a submapping approach as well, and if so how does it work?

The baseline method does not utilize a submap-based approach for identifying subgoals and planning. However, as mentioned in Appendix A.1, FARMap can be combined with any planners that employ graph- or submap-based approaches for path planning. For instance, GBPlanner uses a frontier method to select a subgoal, while employing a graph-based approach for rapid path planning. It can be integrated with FARMap by replacing the Planner component (such as Dijkstra’s algorithm or RRT) and the subgoal identification mechanism (Frontier) with those from GBPlanner.