Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks

We thank the referee for constructive comments, especially the suggestion of highlighting the relevance of our work for the neuroscience community. We addressed raised concerns by performing new analyses and improving the text, as we describe below (submitted as two comments).

• Relevance to neuroscience:

Thank you for recognizing the neuroscientific interest in our results; we are primarily motivated by insights from and for neuroscience. To make this more clear, we added a new section 5 on the neuroscientific implications of our findings.

• Single neuron vs network timescale:

We agree with the referee; this is a fascinating point, and we should expand on it and highlight the possible biological implications. Our results indicate that adapting single-neuron timescales to the task requirements is less robust than learning based on adjustment of synaptic weights. This is greatly in line with how the neural networks in the brain primarily train: although there are possibilities to adjust single-neuron timescales, primary learning is implemented in adjusting the connectivity. However, the heterogeneity in single-neuron timescales might be very important in the brain and our networks. We changed the duration of each stimulus presentation and found that the individual timescales track the input timescale in single and multi-head networks (Appendix C). This proposes one possible mechanism for how single-neuron timescales (equivalent to the membrane constant in our formalism) might be important for computations in reality, where the input timescales could vary a lot, e.g., different input timescales might be tracked by neurons with different taus.

• Short-term plasticity and adaptation:

Indeed, STD/STP and adaptation are important in biological networks. Here, we simplify the biological picture for the sake of having only two dominating timescales, however, the inclusion of such mechanisms would introduce additional timescales generated by the plasticity/adaptation. From our results, we would expect that matching these timescales to the input could, for example, change which tau’s are optimal. In biological networks with given adaptation or STD/STP mechanisms, we believe that the learned recurrent connectivity will still be the defining factor for the resulting timescales, but expect more complicated interaction between different mechanisms. Moreover, previous work suggests that single-neuron adaptation timescale (Beiran & Ostojic, PLoS CB, 2019) does not affect collective network timescales that are probably more relevant for computations. We added this discussion to the limitations section and included the suggested references.

• Task choice and generalization:

We agree with the referee and weakened our statements and highlighted this in limitations.

• N-DMS and parity:

The difference between these two tasks is that N-DMS only requires memory of input, whereas N-parity also needs computation (though quite simple). We add this briefly to the text (section 3.1).

We thank the referee for a thorough review and appreciate the positive feedback and finding our results promising. We address the questions raised below.

• Memorylessness at tau=1:

$\tau = 1$ makes individual neurons memoryless in the absence of any network interactions. However, the network as a whole is not memoryless, and recurrent inputs to each neuron create correlations between the activity of neurons across different time steps (as one example, you can imagine the wave of activity propagating through the network in circles). That means when $\tau = 1$, the memory capacity of the network is fully determined by the connectivity. We now clarify this in the text (pages 2-3).

• Relationship between $\tau$ and $\tau_{net}$

$\tau_{net}$ is generally a function of tau and the recurrent weights. We edited the text and provided a better explanation (page 3).

We thank the referee for constructive feedback and helpful comments. In the following, we go over the questions/ noted weaknesses and describe changes and new analyses.

• Overstatement of the results:

We edited the paper and made our statements more grounded and concrete (e.g., Relevance to Neuroscience, Related Work sections) and extended the limitations section

• Additional experiments:

While we could not generate an entirely new set of experiments in a short time, we propose the general set of requirements these experiments need to satisfy (in the limitation section). Moreover, we described an additional task that would fulfill these requirements (Appendix D).

• Significance of single-head vs multi-head perturbation and retraining (former Fig.8):

We run the significance test for different perturbation strengths and retraining N (Appendix M). For all perturbations, when the perturbation size is very small, there is almost no difference; for larger perturbations, there is a significant difference, and then performance for both networks degrades to the level where they are indistinguishable again.

• Clarity:

We edited section 4 to have paragraph headings and clear claim-based summary sentences, modified figure captions and introduced time lag earlier.

• Different experimental settings/ testbeds:

We added to the limitation section a requirement for the task. Moreover, we added Appendix D, discussing a possible task for future research: pattern generation, which is a common task. We adapt this continuous-time task and turn it into a curriculum for generating the sum of sine waves.

• Relevance for neuroscience:

We see our work as strongly motivated and interpretable in the neuroscience context. We added a new section 5 to better discuss this relevance. In brief, our results can be extended to continuous-time settings relevant to modeling brain networks. Moreover, we discuss that using recurrent connectivity to learn the tasks and develop long timescales aligns well with previous experimental findings.

We thank the referee for motivating questions and suggestions. In the following, we address noted weaknesses and suggestions.

• Editing for colorblind:

Thank you for the suggestion. We changed the figures such that in the panels where the single- and multi-head lines for different experiments are present together, the multi-head lines are dashed (Fig. 3b, Fig. 4b). In all other cases, we checked in Black and White that the lines are distinguishable.

• Figure S12:

Thank you, we implemented your suggestion (new Fig. S2).

• Figure 6:

Indeed, we fitted the linear relationship only for small N (N =20). To make it easier to identify, we added the lines as a guide for the eye. The reason for this is to show that the scaling of the dimensionality in the N-DMS task is not linear: for the points with N>>20, the line does not fit, and growth is slower. In the updated version of Fig.6 and new Fig.S20, we present the fits for different ranges.

• Figure 8c (now 7e):

We now include the chance-level accuracy in the panel. We retrained the networks for higher N and observed for both single and multi-head curricula, a convergence towards the chance level. Moreover, we calculated the p-values and presented them in Appendix K to demonstrate that there is a significant difference between single- and multi-head networks.

• Relation to the previous work and limitations:

We substantially edited the related work and limitations sections to highlight the significance of our findings in comparison to previous works and its potential extensions for future studies. In brief, although there were theoretical studies analytically deriving timescales for simple networks, those studies were limited to the specific (and simplistic) connectivity and investigated only the relationship between the topology features and local timescales, often without discussing their functional relevance. Here, we extend those findings and relate the dynamical properties directly to learning and computations relevant to working memory tasks, bridging theoretical mechanistic concepts from computational neuroscience to functional concepts in ML. We also show that the multi-head curriculum improves not only the training of RNNs but also of GRUs and LSTMs.

Questions:

• Convergence:

The equation is $r(t) =$ (1 - 1/τ) \cdot r(t-1) + \sigma(\text{Input}) / \tau $_a$

If $\tau$ → ∞ , then $1/\tau$ → 0, so the equation becomes:

$r(t) = [r(t-1)]_a$

If $r(0) \geq 0$, then r(t) = r(0) for all t (since $[x]_a = x$, for $x\geq 0$ and $[x]_a = a \cdot x$, for x < 0 )

If $r(0) < 0$, then $r(t) = a^t \cdot r(0)$ →0, since $a << 1$

Thus as $\tau$ → ∞, we have convergence

• Model selection:

Thank you for the suggestion of the relevant literature for the choice of model selection criterion. We referred to it and also tested BIC and found that there is above 95% agreement between AIC and BIC (Fig. S1).

• Figure 4 (meaning of k):

Thank you for noting, we edited the caption and added a more detailed explanation about k interpretation in new section 5 and Appendix C.

• Training multi-N without curriculum statement:

We rewrote it as: “Furthermore, training directly on a multi-N task without using an explicit curriculum results in the emergence of the multi-head curriculum: networks learn first to solve small-N tasks and then large-N tasks (Appendix I), supporting the use of the multi-head step-wise strategy.”

• Figure 5 - Behavior of timescales for large N:

For large N, the network may need strategies that go beyond increasing the timescales of individual neurons (measured by tau and tau_net). These solutions possibly employ coordination across multiple neurons to generate slow dynamics on the population level rather than the activity of individual neurons. As a proxy for such coordinations, we measure the timescale of population dynamics (summed activity of all neurons) tau_pop and show that for all tasks and curricula, tau_pop grows with N (Fig. S4).

• Meaning of the strong inhibitory weights:

In the single-head curriculum, the long taus make the influence of recurrent and external inputs larger (equ. 1). Thus, a perturbation can lead to a very large response, if inhibition does not stop it. We illustrated this stability issue by perturbing the inhibitory weights (Appendix J., Fig. S19). For single-head networks, decreasing the inhibition leads to the rapid explosion of network activation, whereas for multi-head networks changes are minimal.

• Retraining:

Indeed, we took 20 epochs for retraining. The reason is to capture the speed of convergence, which would indicate the benefit the training has from the initialization with a network trained for N=16 (single-head), or N=2,..., 16 (multi-head). Generally, many networks can train to solve N=17 from scratch, so both initializations would converge to the perfect performance if given enough time. We tested retraining for 50 epochs, which produced qualitatively similar results.