We thank reviewer 2ZH1for the review and the valuable suggestions. Our responses to the raised points are the following:

The theoretical framework proposed, while enlightening the UBLA and BLA models, does not lead to a concrete prescription as to how to build RNNs that are robust to weight perturbations.

There is some literature on the necessary conditions for attractive and bounded solutions to exist, see for example Yi (2001). See conditions for a line attractors in ReLU RNNs in Yu (2009), one just needs to rotate these line attractors so that they become bounded. Furthermore, conditions of the equilibria in more generality (line and plane attractors) of the 3D RELU network are presented in Xiang (2021) and parameters for bounded plane attractors for 4D tanh networks are discussed in Yu (2023). These construction should lead to RNNs with continuous attractors that are robust to weight perturbations given our theory.

how do the findings extend to larger models?

The theory tells us that regardless of the dimensionality of the RNN, as long as they have bounded normally hyperbolic continuous attractors the only way to create exploding gradients is if there is a chaotic system in the neighborhood of the original. Currently, all systems considered have a slow flow after perturbation, which can be maintained with the correct learning rate. If one would take non normally hyperbolic attractors, the exploding gradients would form an even bigger problem. For RNNs with a ReLU activation function, all continuous attractors are normally hyperbolic however, whether they are bounded or not.

to have robust RNNs are to pick ones which have bounded and smooth stable manifolds

This indeed underlies the generality of the theory. However, it is not required that the invariant manifold be smooth. Furthermore, using backpropagated gradients would be one way to counteract noise, which is especially problematic for non-normally hyperbolic invariant manifolds.

The experimental setup may not be relevant to larger models, given that the RNN robustness is studied when the model is already initialized at its optimum, which may not be the case for a realistic task.

We agree that our theory only covers cases where there is a continuous attractor implemented in an RNN. However, our theory does apply to RNNs of all sizes, hence also for larger models. these theoretical results nullify the concern of the fine tuning problem in neurosciences We agree with the reviewer that this is too strong of a claim given our current understanding of continuous attractors and we have softened our language in the manuscript accordingly. However, we would still like to note that the BLA has the broadest range of learning rates that are optimal and far away from really badly exploding gradients. Furthermore, the somewhat negative results are specific for the backpropagated gradient that we consider here. It is not a general principle that homeostatic restoration of continuous attractors is challenging, but only for the gradient signal considered in the paper. In fact, a control mechanism could be theoretically devised that relies on finite control input for all bounded continuous attractors, while this is not guaranteed for unbounded continuous attractors.

initialization strategies for RNNs in ML

Further research needed to test efficiency of BLA as an initialization strategy. Our theory suggests that networks with bounded continuous attractors) will not exhibit exploding gradients under noisy conditions, while RNNs initialized with the identity matrix will.

Thank you for your time and consideration.

Yi, Z., Tan, K.K. and Lee, T.H., 2003. Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 15(3), pp.639-662.

Yu, J., Yi, Z., & Zhang, L. (2009). Representations of continuous attractors of recurrent neural networks. IEEE transactions on neural networks, 20(2), 368-372.

Xiang, W., Yu, J., Yi, Z., Wang, C., Gao, Q., & Liao, Y. (2021). Coexistence of continuous attractors with different dimensions for neural networks. Neurocomputing, 429, 25-32.

Yu, J., Chen, W., Leng, J., Wang, C., & Yi, Z. (2023). Weight matrix as a switch between line attractor and plane attractor of ring neural networks. Neurocomputing, 521, 181-188.

We thank reviewer EwtU for the review and the valuable suggestions. Our responses to the raised points are the following:

i. the lack of mathematically precise explanations

We have adjusted the language in Theorem 1 (Fenichel's Invariant Manifold Theorem) to make the statement mathematically mode precise. To comment on the complications of determining the extent of noise that a given system can handle, a lot more research is needed to develop a full account for what the maximal $\epsilon$ (the timescale parameter) value is for which persistence holds. But in principle it can be calculated, see Chapter 6.5 in Wiggins (1994). If there are any particular parts of the manuscript that the reviewer found imprecise, we would be happy to know so that we can improve the overall mathematical precision.

ii. the lack of guidance as to how to interpret the 2D results for RNN training in general

Normal hyperbolic attractors are persistent in arbitrary dimensions. This is demonstrated for a higher dimensional example of the head direction network implementing a ring attractor. All perturbations in the vicinity of the original system lead to a bounded invariant manifold, for which we describe the Morse decomposition in terms of the fixed points and their connecting orbits. Saddle nodes are always connected to stable fixed points in the direction of their unstable manifold.

Does this mean that the attractors of the hidden states are never chaotic attractors?

For 1 and 2 dimensional continuous attractors, small enough perturbations never lead to chaotic attractors, because the perturbed invariant manifold needs to be homeomorphic to the original continuous attractor and there can be no chaos in dimensions 1 or 2 (see for example the Poincaré–Bendixson theorem). For higher dimensional continuous attractors this is not guaranteed, however, the chaotic flow will be slow according to the theory.

While the authors mention that these theoretical insights can be used to guide RNN training dynamics away from problematic bifurcations, it is unclear how that can be accomplished in practice, in high-dimensions.

Normal hyperbolic attractors will be persistent in arbitrary dimensions, even in PDEs. For ReLU RNNs, there is some literature on the necessary conditions for attractive and bounded solutions to exist, see for example Yi (2001). See conditions for a line attractors in linear threshold RNNs in Yu (2009), one just needs to rotate these line attractors so that they become bounded. Furthermore, conditions of the equilibria in more generality (line and plane attractors) of the 3D RELU network are presented in Xiang (2021) and parameters for bounded plane attractors for 4D tanh networks are discussed in Yu (2023).

We will include these in the manuscript. However, for RNNs with arbitrary activation functions, a general theory is still lacking.

When can RNN dynamics be decomposed into slow-fast systems for which we can apply Fenichel duality?

For continuous attractors this decomposition is always possible. We would kindly bring to the attention of the reviewer that Fenichel’s Invariant Manifold Theorem should not to be confused with Fenchel’s duality theorem (https://en.wikipedia.org/wiki/Fenchel%27s_duality_theorem). For other systems, decomposability into fast and slow subsystems needs to be investigated on a case-by-case basis.

When does the input dynamics (discrete-time map) induce Fenichel-type attractors in the dynamics of the hidden state (may be the authors can expand a bit on their observations about the loss landscape near the normally hyperbolic fixed point).

We show some results of RNN trained on an integration task that show that bounded normally hyperbolic invariant sets (is this what the reviewer refers to as Fenichel-type attractors?) develop during learning and that even with (constant) input the system exhibits Fenichel-type attractor behavior, typically in the form of limit cycles and fixed points. These results are included in Supp.Sec. 12 (Supp.Fig. 20.

Thank you for your time and consideration.

Yi, Z., Tan, K.K. and Lee, T.H., 2003. Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 15(3), pp.639-662.

Yu, J., Yi, Z., & Zhang, L. (2009). Representations of continuous attractors of recurrent neural networks. IEEE transactions on neural networks, 20(2), 368-372.

Xiang, W., Yu, J., Yi, Z., Wang, C., Gao, Q., & Liao, Y. (2021). Coexistence of continuous attractors with different dimensions for neural networks. Neurocomputing, 429, 25-32.

Yu, J., Chen, W., Leng, J., Wang, C., & Yi, Z. (2023). Weight matrix as a switch between line attractor and plane attractor of ring neural networks. Neurocomputing, 521, 181-188.

We thank reviewer yzhP for the review and the valuable suggestions. Our responses to the raised points are the following:

1. In this paper we focus on the properties of continuous attractors that might be necessary for some specific tasks, and are not concerned with learning chaotic dynamics. Furthermore, unstable dynamics won’t be present in our proposed RNNs as we are concerned with bounded continuous attractors that will have a persistent attractive invariant manifold. We clarified this in the paper on page 3 at the beginning of Section 2.

2. We agree that the paper only covers one specific scenario (that of bounded invariant manifolds in RNNs) and may not cover the broader spectrum of possibilities. However, note that our theory is applicable to all RNNs and biological recurrent networks that can realize such invariant manifold structures. We believe that it is this generality of the theory that makes it relevant to the broader theoretical neuroscience and machine learning communities.

3. Smoothness of the system under consideration is indeed a valid worry in general. However, for continuous piecewise linear (PWL) dynamical systems for our purposes of stability of the persistent invariant manifold, the theory is applied (ReLU RNNs are continuous). If the critical manifold is globally stable then the system is forward invariant in a neighbourhood of the critical manifold, and Fenichel’s invariant manifold theorem is applicable (Simons, 2018). We believe that the theory can be applied to PWL continuous systems with continuous attractors in high generality because of their global attraction property, see for example the analysis of the head direction model in Section 3.2. For remarks on the application of the fast-slow theory to PWLs, see Theorem 19.3.3 in Kuehn (2015). And in practice we see that the continuous attractor persists as an invariant manifold, see the bifurcation analysis both on the bounded line attractor network and the head direction representation network. For Filippov systems the situation is different because they are not continuous. For fast-slow analysis of Filippov systems, see Cardin (2013), however, note that these are outside the scope of our investigation. We included a remark on the applicability of Fenichel’s invariant manifold theorem to continuous piecewise linear (PWL) dynamical systems on page 4 in the second paragraph. This means that even for PWL continuous dynamical systems, the normally hyperbolic invariant manifold that is the continuous attractor is going to persist. It will not maintain smoothness because $r<1$, but we don’t think that causes an issue for the (biological) systems that implement continuous attractors. If it is important that the invariant manifold persists in a smooth way, namely that the perturbed invariant manifold is $C^r$ then other activation functions need to be used.

4. We thank the reviewer for this suggestion and will incorporate it in the text as one way of finding maximal epsilon bifurcation parameters around continuous attractors. Our theory shows that for 1 and 2 dimensional continuous attractors, there are no exploding gradients for small enough perturbations, even though bifurcations do occur. The only possibility is the creation of chaotic orbits, which could still cause a problem. This paper however is not considering bifurcations involving chaotic behavior, so we don’t see the relevance of considering this part of the literature. Note finally that while we acknowledge the relevance of this recent paper to our research, it was published after the submission of our manuscript.

Thank you for your time and consideration.

Cardin, P., Da Silva, P., & Teixeira, M. (2013). On singularly perturbed Filippov systems. European Journal of Applied Mathematics, 24(6), 835-856.

Kuehn, C. (2015). Multiple time scale dynamics (Vol. 191). Berlin: Springer.

Simpson, D.J. (2018). Dimension reduction for slow-fast, piecewise-smooth, continuous systems of ODEs. arXiv preprint arXiv:1801.04653.

We thank reviewer K1Kz for the review and the valuable suggestions.

Before addressing the comments of the reviewer, we would like to make a statement on the generality of the approach. While we acknowledge the importance of practical validation, we firmly believe that our work holds substantial merit even in the absence of these experiments. Allow us to elaborate on the theoretical progress we aim to contribute to the field. Our study focuses on a fundamental theoretical insight that transcends the immediate realm of practical applications. We establish that, under the condition of attractors being normally hyperbolic, they exhibit a robustness to small perturbations, thereby retaining their attractiveness as a set. This finding not only adds a valuable theoretical dimension to our understanding of machine learning but also has broader implications for the advancement of the field.

In essence, our work strives to unveil general principles that govern the stability of attractors, contributing to the foundational knowledge of machine learning dynamics. While we acknowledge that practical applications may not be readily implementable, we argue that the theoretical framework we propose lays the groundwork for future innovations and discoveries.

We invite the reviewer to consider the broader impact of our theoretical contributions, which extend beyond the confines of specific experiments. By elucidating the stability properties of attractors, our work opens avenues for further exploration and application in diverse contexts. We remain committed to advancing the field of machine learning and believe that our theoretical findings provide a solid foundation for future research endeavors.

Our responses to the raised points are the following:

are the bounded attractors more likely to be learned from arbitrary initialisations

We agree with the reviewer that bounded attractors should be easier to learn, however, only if the network dynamics is already near the solution. From an arbitrary initialization, we find that various slow manifold solutions that approximate a continuous attractor can be reached. In our numerical experiments, when circular variables were trained, a ring-like structure often emerged. We added these results to the supplementary material in Supp. Sec. 12. We furthermore present some preliminary results on training RNNs on the same integration task, but from arbitrary initialization for the parameters. In this case, we show that bounded attractors are more likely to be learned if the task duration is long enough.

What happens in complicated situations?

We will try to clarify what could happen in an example of a task that is more complicated. We will reflect on RNNs trained on multiple tasks, such as in Driscoll (2022). In this case, one would need to study how certain invariant manifolds used by one task are influenced by training the system on another. One could then another and then to compare different implementations of a line attractor (UBLA/BLA) while training the system on two different tasks, an integration task and a perceptual decision-making task for example.

In the supplementary, we now show some results on training RNNs on an angular velocity integration task (for which continuous, circular variables need to be integrated and maintained by the network). In this case, the trained RNNs exhibit a ring-like structure.

If the reviewer had other complicated situations in mind, we would be happy to address what happens in them.

What does it mean for biological networks?

As suggested in the paper, structural constraints (such as Dale's law and mutual inhibition which are both satisfied for the bounded line attractor) could contribute to biasing learning towards bounded attractors.

Can any insights be made into the sorts of sequence tasks that machine learners care about?

Our theory suggests that compact continuous attractors are more robust under noisy and continual learning. For high-dimensional RNN training for general tasks (e.g. sequential MNIST), as long as these involve the need to represent continuous variables, continuous attractors will be a natural solution, see for example Dehyadegary (2011).

Many thanks for your responses.

I still think the result and potential avenues are interesting, but, while I appreciate the additional results, I still think that much more needs to be done to actually prove your points. I have nevertheless raised my score to reflect the addiional results.