Dynamical phases of short-term memory mechanisms in RNNs

We thank the reviewer for their constructive feedback, as well as their recognition of our methodological rigor and insightful contributions to understanding short-term memory mechanisms in artificial networks. Please find our responses below:

Q1 While learning rate is an abstract optimization hyperparameter, prior literature indicates that it can be influenced by both external and internal biological factors. See below for a brief discussion that we will add:

Interestingly, our findings that longer delays require smaller learning rates parallels observations in neuroscience, where tasks with longer temporal gaps between cues and outcomes often pose greater credit assignment challenges. During cue-reward delays, dopamine activity has been suggested as a mechanism for solving credit assignment through eligibility traces. For instance, synaptic-level eligibility traces have been evidenced by dopamine's role in potentiating synaptic changes within precise temporal windows (Shindou et al., 2019; Yagishita et al., 2014). Additionally, dopamine may facilitate credit assignment through activity ramps that increase as subjects approach rewards (Fiorillo et al., 2003, Mikkael et al., 2022, Krausz et al., 2023). But more explicitly, a recent study (Coddington et al., 2023) showed that dopamine can directly modulate learning rates to support effective decision-making. Similarly, our results align with studies showing that the duration of the inter-trial interval—analogous to our post-reaction time—can also affect learning rates. Specifically, another neuromodulator, serotonin, has been found to modulate learning rates following long inter-trial intervals (Iigaya et al., 2018).

Thank you for this exciting connection that significantly improves the impact of our work for the neuroscience community.

Q2 We appreciate this insightful question. Our task was intentionally designed to isolate memory maintenance mechanisms, independent of representational content, to expose the underlying structure of the solution space (e.g., phase transitions). In tasks like delayed cue discrimination (Fig. 5), inputs determine the initial condition of the latent state, which then evolves along either a slow manifold or a limit cycle. In both scenarios, the identity of the cue is encoded in the trajectory, e.g., distinct slow-point manifolds.

How feature representations are integrated into these mechanisms is a compelling direction for future work. For instance, one could introduce a contextual variable that modulates the delay period for the same stimulus. This would allow investigation into whether the RNN reuses its existing slow point or limit cycle structures, or develops new attractors for each context. We view questions like these as natural next steps—and our publicly available dataset of trained RNNs is well-suited for such follow-up studies. We will add a paragraph to the outlook about this interesting future direction.

Weakness: hyperparameter sensitivity and generalizability In the revised draft, we include a new experiment in which we double the neuronal time constant $\tau$ (one of the hyperparameters in the RNN architecture). The results remain virtually unchanged, with the same phase diagram emerging. We will report these findings in a supplementary figure. We also note that a recent work (Park et al. 2025; ICLR 2025) has empirically found a similar phase diagram of algorithmic strategies, though in a completely different context.

While testing all hyperparameters is computationally infeasible given the scale of our experiments, we now include the following paragraph in the Discussion to explicitly state this limitation and outline future directions:

In this work, primarily constrained by the immense compute required (about a month of a standard GPU time), we fixed architectural parameters and activation functions, which is a limitation of this work. Future work could explore the effects of different activation functions, network size (although recent results suggest that larger networks may reduce efficient learning rates; see Dinc et al., 2025), self attention-like mechanisms, and short-term synaptic plasticity on the emergence of different memory strategies. We conjecture that, since the toy models highlight the fundamental constraints associated with the latent dynamical systems learned within these architectures, phase diagrams we observe in this work may present universally across models (see also Park et al., 2025).

Generalizability and relevance of the toy models We agree that this is central to the broader impact of our work and should have been stated more clearly. Please also see our response to Reviewer CYoj.

Final remarks Please let us know if any further clarifications are needed. We sincerely appreciate the reviewer’s feedback and support. We agree that stronger alignment with neuroscience is important and have made necessary revisions.

We thank the reviewer for their thoughtful comments and highlighting our approach's strengths—especially our effort to simplify a complex problem into an interpretable framework. Their suggestions have greatly shaped our revisions. Below, we address all specific concerns and weaknesses.

Q1 That is an excellent point. As the reviewer notes, models based on dynamic synaptic variables like STSP represent a fundamentally distinct mechanism from the activity-based attractor models we focus on. In an earlier draft, we mentioned both classes—such as Mongillo et al. (2008)—and then narrowed the focus to activity-based models (Fig. 1). Based on the reviewer’s feedback, we restored and expanded the neuroscience context. In addition to intro, discussion mow highlights that STSP mechanisms may allow networks to bypass the learning constraints we identify for fixed-weight attractor models (e.g., Masse et al., 2019). The Conclusion links to recent AI models like Mamba, which integrate memory via local mechanisms that may functionally resemble STSP.

Q2 This point is addressed in our reply to Reviewer CYoj.

Q3 A key motivation for releasing all trained models is to enable reuse without retraining. We now clarify that the dataset required ~1 month of GPU time. Due to space constraints, we briefly outline three research directions this dataset enables (with more to be added in the Outlook):

• Curriculum learning: Training pre-trained models for longer delays with reduced learning rates may enable faster convergence and offer insight into curriculum/transfer learning. The toy models could support new curriculum strategies with theoretical guarantees—relevant for experimentalists (including co-authors) who find it difficult to train mice on seconds-long delay STM tasks.

• Model initialization: Exploring how weight initialization affects convergence to slow-point vs. limit-cycle strategies. For example, initializing near an existing solution (e.g., SP) may bias learning toward that strategy and potentially shift phase boundaries (Fig. 5).

• Robustness: Comparing robustness by perturbing neurons (e.g., mimicking optogenetics) may reveal key differences (oscillations vs decay) between slow-point (SP) and limit cycle (LC) strategies.

We also refer the reviewer to our response to Reviewer 59aj’s second question for more on the dataset’s utility.

Q4 Testable predictions. Our work predicts that both SP and LC dynamics can generate sequential neural activity, but differ in structure: SP dynamics converge to a slow-point attractor by trial end, whereas LC dynamics continue producing trial-type-specific sequences beyond reward. This distinction is testable by analyzing neural activity after task completion. While Rajan et al. (2016) explored sequences in RNNs, we link them to distinct attractor types. For instance, Fig. 4 predicts that extending the post-response period biases dynamics toward LC (oscillatory) or SP (ramping) solutions. Although this period can’t be fully removed in experiments, it can be disrupted—e.g., via bulk optogenetic inhibition of task-relevant regions, providing a plausible way to test this prediction.

Relation to recent studies: We will cite the suggested works and place greater emphasis on neuroscience connections (previously omitted due to space). The reviewer’s interest—as a neuroscientist—strongly encouraged this expansion.-

Q5 We thank the reviewer for raising this thought-provoking possibility of a third dynamic regime—one in which a network operates in a limit-cycle (LC) mode during the task but undergoes a modulatory shift in excitability toward the end of the trial that disrupts the remaining cycle, effectively mimicking a slow-point (SP) solution in the end. This is indeed how we would expect LC solutions to be utilized in practice. Distinguishing such a modulated LC from a true SP is an exciting experimental challenge and could reveal important aspects of synaptic learning rules and neuromodulatory control. We outline three levels of experimental inquiry to test this hypothesis in vivo (briefly):

• Exp 1: The assumption of functional reorganization can be tested by combining constant-power bulk optogenetic activation with - calcium imaging at different trial stages.
• Exp 2: Genetically encoded fluorescent sensors (e.g., dLight, GRAB-ACh) can be used to track neuromodulatory signals during task performance. A signal peaking in the post-response window would support the proposed mechanism.
• Exp 3: To establish causal roles, one could block or genetically delete the relevant receptors and observe changes in post-trial neural dynamics or task performance.

Final remarks Thank you for the style suggestions and pointing out the typos. We incorporated all suggested edits and reviewed the manuscript for clarity, consistency, and polish. Please let us know if any points need clarification. We sincerely appreciate the reviewer’s enthusiasm and attention to detail.

We thank the reviewer for their thoughtful comments and for recognizing that both the theoretical and empirical components are well-executed. We understand the main concerns to be (1) the clarity of the connection between theory and experiments, and (2) the perceived novelty of our contributions. We have revised the manuscript accordingly and address both below.

(1) Due to space constraints, we had to condense this connection in the original submission. Here, we clarify the rationale and why toy models are directly relevant for the RNNs.

In short, consider the models in Eq. (S1) and (S15) of the main text. These approximate the latent trajectory of a rank-one and rank-two RNN around a local minima, respectively. The scaling laws, derived from these models, accurately predict the transition boundaries observed in large-scale RNN training (new result, $\beta = 4.05 \pm 0.1$, $R^2 = 0.99$), when RNNs are learning those particular solutions in their latent subspaces.

Here, it is worth noting that we empirically tested the scaling laws in the (realistic) full-rank RNNs, whereas the toy model theoretically models the latent trajectories in the low-rank RNN. The theoretical connection is an open question in this field and is a current hot topic (Valente et al., 2022). However, Valente et al. 2022 has empirically shown that trajectories of full-rank RNNs trained on behavioral tasks can often be reproduced by their low-rank counterparts that are sufficient to solve the task. In our case, rank-one RNN is sufficient to solve the task with a slow-point manifold, so it is not surprising that the toy model predictions explain (qualitatively) the existence of and (quantitatively) the boundary scalings of the phase diagram.

Longer rationale justifying, e.g., why the slow point model is a realistic approximation:

• RNNs are universal approximators of dynamical systems, and the rank of their recurrent weight matrix determines the dimensionality of the latent dynamics they can implement (Beiran et al., 2020).
• Recent work has shown that even complex tasks can be solved with low-rank RNNs (Dubreuil et al., 2022), and that full-rank RNNs are often effectively low-rank in practice (Valente et al., 2022). In our empirical study, we similarly observe that both low- and full-rank RNNs converge to low-dimensional solutions, which take the form of either slow-point manifolds or limit cycles.
• The slow-point toy model is the normal form (for reference, see Nonlinear Dynamics and Chaos by Strogatz) for a saddle-node bifurcation, which approximates one-dimensional dynamical systems around their local minimum (slow-point formation), including those implemented by the rank-one RNNs in Eq. (2).
• Finally, learning the weights in rank-one RNNs implementing a slow-point corresponds to varying the parameter of this normal form, which gives rise to the phase diagrams derived in Fig 4C (and observed in full-rank RNNs in Fig 6).

We will briefly incorporate this explanation into the revised manuscript, with the rigorous mathematical details in the supplementary.

(2) We agree with the reviewer that studying attractors as short-term memory mechanisms in RNNs is one of the oldest problems in computational neuroscience. However, our contribution is not the problem itself, but our novelty lies in findings and mechanisms that were previously not known in this long-standing field. To be specific, here are a few of the novel findings in our work:

• To our knowledge, this is the first study to systematically analyze how latent mechanisms emerge during learning (e.g., such a description does not exist for the STSP based models, also cited by the reviewer).
• We identify a sharp transition between slow-point and limit-cycle solutions depending on task delay and learning rate. To our knowledge, we are the first to show that there is a phase diagram of strategies (w.r.t., optimization and task parameters).
• Unlike prior work, which often focuses on complicated working memory tasks (e.g., saccades in the references cited by the referee), our custom task design focuses on short-term memory and balances simplicity and expressiveness, allowing us to analyze dynamical structures at scale.
• We show that small changes (e.g, a post-response) can qualitatively shift the memory strategy learned.
• Our large-scale dataset (35,000+ trained RNNs, first dataset of its size reported and to be made public) will be released to support further studies on training dynamics, robustness, and learning strategies.

Misc We thank the reviewer for pointing out the omission of synaptic models of working memory, which we now explicitly cite. Please also see our response to Reviewer pHmu. We also see that you asked for an ethics review. How could we address your concerns?

Final remarks We hope these clarifications address the reviewer’s concerns and are grateful for the feedback, which significantly improved the paper’s presentation and scope.