Unconditional stability of a recurrent neural circuit implementing divisive normalization

Weaknesses:

• “the introduction of the dynamics … why they are important”. Answer: We thank the reviewer for the insightful feedback. We will add more intuition about the dynamics in the model description and compare ORGaNICs to LSTM, an RNN architecture similar to ORGaNICs. We will also provide intuitive definitions of the terms used in the stability proof.

• “in L43, it is written that stability … Zucchet and Orvieto, 2024”. Answer: We appreciate the referee's observation. While stability indeed addresses the issue of exploding gradients, excessive stability—where the real parts of the eigenvalues of the Jacobians are significantly smaller than 0—can lead to a lossy system prone to vanishing gradients. ORGaNICs effectively mitigates both issues: Exploding gradients: through its inherent stability. Vanishing gradients: by processing information across various timescales (and via the long effective time constant when the membrane time constants are fixed) while maintaining stability, resulting in a blend of lossy and non-lossy neurons. The efficacy of ORGaNICs in mitigating vanishing gradients is demonstrated by its competitive performance against architectures specifically designed to address this issue, such as LSTMs. We will provide a more detailed explanation of this in our revised manuscript. Moreover, we posit that the built-in normalization feature of ORGaNICs may alleviate the "curse of memory" described by Zucchet and Orvieto (2024), as normalization is proposed as a potential solution to this problem.

• “in L61, … stability of the system”. Answer: Divisive normalization (DN) was introduced as a model of steady-state response of neurons, functioning as a static nonlinearity similar to batch and layer normalization. However, in the brain, there are no static nonlinearities; it is proposed that DN is achieved via a recurrent circuit. ORGaNICs is such a recurrent circuit designed so that the responses of neurons at steady state follow the DN equation, with stability being an emergent property of the circuit. The DN equation indeed generalizes batch and layer normalization, as shown in the work by Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016).

• “in L349, … made more precise”. Answer: The key similarity is that both operate via changing the input gain, which also models a wide range of results from neurophysiological and psychophysical experiments. In particular, it has been shown that normalization explains experimental measurements of temporal attention (Denison, Carrasco, Heeger, Nature Human Behavior, 2021), i.e., over the time axis, analogous to transformers.

• “the kind of neural network … additions to Section 6.1”. Answer: We thank the reviewer for pointing this out. We checked if the models dynamics are stable throughout training and found that this is indeed true. Please see Fig.2 of the attached pdf.

• “in Section 6.2, … of the performance”. Answer: We are aware that state-space models perform better at these tasks and we will mention this (along with citations) in the paper, but our goal is to compare ORGaNICs with other RNNs designed with the property of less-dissipation and stability to mitigate the problem of vanishing and exploding gradients. We thank the reviewer for pointing out the sensitivity of initialization on sequential modeling tasks. We have added that detail (uniform random initialization) in the revised manuscript.

Questions:

• “The model … E neurons?” Answer: Experimental data suggest that the ratio of E and I neurons is 4:1. Our results are also applicable for a different E/I ratio than 1.

Weaknesses:

• “Fundamentally, ORGaNIC … one particular effect found in V1”. Answer: DN is observed in numerous cortical areas beyond V1 and across different species (Carandini & Heeger, 2012). It explains a wide range of experimental phenomena in various neural systems and cognitive processes (citations in lines 49-53). DN can also be shown to be the appropriate nonlinearity for removing dependencies in sensory inputs, resulting in a set of neural responses that are statistically independent (Simoncelli and Olshausen, 2001). Consequently, DN is not just an effect observed in V1, but a computational principle (Carandini & Heeger, 2012) that can be derived from a statistical optimization criterion. It is thus unsurprising that DN has been shown to dramatically improve the performance of ML models in several applications (e.g., Balleé, Laparra, Simoncelli, 2015; Balleé, Laparra, Simoncelli, 2017). Moreover, batch and layer normalization (of which DN is a generalization, see Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016)) are critically important architectural motifs in modern deep learning architectures. There is thus overwhelming evidence that DN is an essential feature of both natural and artificial neural systems. In this paper we demonstrate that DN can be integrated into the dynamics of an RNN, leading to a provably stable circuit for which an interpretable normative principle (Lyapunov function) can be derived. This work does not just propose an architecture, but provides 1) unprecedented insight into the impact of normalization on the dynamics of RNNs, through an interpretable normative principle; 2) establishes an important precedent for how the incorporation of neurobiological principles can drive advances in ML.

• “The authors claim … aware of this but not completely”. Answer: In RNN architectures derived from discretized ODEs, such as ORGaNICs, the stability of the dynamical system is intrinsically linked to the vanishing and exploding gradients (VG and EG) problem. Mathematical analysis and details can be found in, Haber and Ruthotto (2017); Chang, Chen, Haber, and Chi (2019); and Erichson et al. (2020). ORGaNICs effectively addresses both problems: EG: Mitigated through the architecture's inherent stability. VG: Addressed by processing information across various timescales (and via the long effective time constant when the membrane time constants are fixed), resulting in a blend of lossy and non-lossy neurons. The effectiveness of ORGaNICs in tackling VG is evidenced by its competitive performance against architectures specifically designed to address this issue, such as LSTMs.

Questions:

• “ORGaNIC begins … name confusing to me”. Answer: ORGaNICs is oscillatory in the sense that it can be mapped to a damped harmonic “oscillator” as we show in the paper. This means that for the right parameters and inputs, stochastically driven ORGaNICs exhibit a peak in the power spectrum, which is consistent with the LFP (Local Field Potential) oscillatory activity observed in neural recordings, even though there are no sustained oscillations.

• “Likewise, can the authors elaborate … it is an extension”. Answer: Gating in ORgaNICs is performed by modulating the input gain ($\mathbf{b}$) and the recurrent gain ($\mathbf{1} - \mathbf{a}^+$). Possible mechanisms of how the responses of these neurons may be computed by loops through higher visual cortical areas and/or thalamocortical loops are discussed in the Discussion/Mechanisms of Heeger, D. J., & Zemlianova, K. O. (2020) and its SI Appendix. The gate in ORGaNICs is an extension of the gates in LSTMs/GRUs because the recurrent gain/gate in ORGaNICs (unlike LSTMs/GRUs) is a particular nonlinear function of the output responses/activation, designed to achieve normalization.

Limitations:

• “In line with my issue … in a single time step”. Answer: The computational cost of ORGaNICs is similar to LSTMs since like LSTMs we have three extra sets of variables ($\mathbf{a}, \mathbf{b}, \mathbf{b}_0$) which are not directly used for prediction. Therefore, ORGaNICs is more computationally expensive than vanilla RNNs. We thank the reviewer for pointing this fact out. We have included this fact in the discussion. However, we do not consider this a limitation, as the increased computational cost results in improved stability, trainability, and interpretability.

• “This work, … neuroscience experiments)”. Answer: Since training biologically plausible neurodynamical models using backpropagation is a challenging task (Soo, W., & Lengyel, M. (2022)), we demonstrate the trainability of ORGaNICs by naive backpropagation and backpropagation through time, without gradient clipping/scaling, on well studied ML tasks. Training ORGaNICs on neuroscience experiments will be done in future work.

• “Also, while this model implements … SSNs (which they cited) can do?” Answer: ORGaNICs has been shown to model a wide range of neural phenomena, including sustained activity, sequential activity, motor preparation, and motor control (Heeger, D. J., & Mackey, W. E. (2019)), as well as simulate the dynamics of V1 activity (Heeger, D. J., & Zemlianova, K. O. (2020); S. Rawat, D.J. Heeger, and S. Martiniani. Cosyne Abstracts 2024) and the emergence of communication subspaces (S. Rawat, D.J. Heeger, and S. Martiniani. Cosyne Abstracts 2023). To address the reviewer’s question, ORGaNICs can do everything that an LSTM can do (while being stable, easily trainable, and interpretable). While a direct comparison of ORGaNICs and SSN across many tasks has not been done, on the one task we consider (static MNIST, following Soo, W., & Lengyel, M. (2022)) ORGaNICs outperforms SSN on the first try (i.e., without hyperparameter optimization). Consistent with our claims, ORGaNICs also performs comparably to LSTMs on sequential tasks such as sequential (and permuted) MNIST.

I thank the authors for the follow up response. I am glad that authors finally understand the position I am coming from, so I will continue with additional comments.

My main point, since the very first review I posted here, is that the tone of the work is a little overblown, citing the following examples:

1. criticizing method papers that are not specific to any architecture to promote your own model (point 1 of my weaknesses that remains unaddressed)
2. absolutely judging every model based on whether it can or cannot do DN (point 2 of my weaknesses that the authors doubled down on in the reply, highlighting the importance of DN)

which leads me to feel like the paper is not coming from a genuine standpoint of trying to contribute to academia, but rather trying to sell itself. Also, it is appalling that the authors can claim "we have no evidence (theoretical or experimental) of additional limitation" when V1 has so many interesting phenomena (see bolded below of biological traits that ORGaNIC does not have). Again, from my purely sincere point of view for this work to read like a good paper, might I suggest a more contributing narrative that fosters collaboration and improvement as a field instead of relying on criticism:

• the vanilla RNN architecture was adapted to be biologically realistic by [Song et al. 2016] by incoporating Dale's Law. [Soo et al. 2024] developed a method for such RNNs to learn long-term dependencies. ORGaNIC is a model that is already built on biological principles, and can learn long-term dependencies intrinsically, therefore not needing any of those results. (those papers provided methods, not models to be compared with ORGaNIC)
• DN is a phenomenon found in the brain, and ORGaNIC is able to express this effect. The biological visual system also expresses other phenomena, such as surround suppression, adaptation, attention-based mechanisms, foveal and peripheral vision, retinotopic mapping, columnar structure, binocular and monocular effects, unique processing of color and plasticity. Other models have been built to study those traits, but here we focus on DN as it is one of the more interesting effects that is believed to stem purely from dynamical effects.

(Edit: To be clear, these are examples of how to write a collaborative narrative. I am in no way forcing the authors to include anything in the paper.)

Once again, I am glad that the authors are now giving genuine limitations, and I hope that they can continue to accept my constructive criticism on the points above. I will reply again right before the deadline to make my final decision.

Weaknesses:

• “Overall this is a … uncertainty rating)”. Answer: We argue that the potential impact is high. LSTMs and GRUs have had a huge impact on ML/AI applications even though they are not always stable and hence require ad hoc techniques for training (e.g., gradient clipping/scaling). Furthermore, although normalization has been shown to improve the training and performance of CNNs, there has been no principled way of adding normalization to RNNs (putting a normalization layer between RNN layers is not the same as normalizing the activations within each RNN layer). ORGaNICs overcomes these limitations, with comparable capabilities to LSTMs and GRUs, with built-in recurrent normalization (motivated by neurobiology) that, as we show, is sufficient to guarantee dynamical stability. Moreover, we note also that normalization is an essential architectural motif in modern ML, and that divisive normalization has been shown to generalize batch and layer normalization, as shown in the work by Ren, M., et. al., (2016). As such, the ability to integrate normalization in the dynamics of an RNN, and to analytically demonstrate the impact of normalization through the derivation of an interpretable normative principle (Lyapunov function) is far from esoteric and very much needed in the theory of deep learning. Finally, our work establishes an important precedent for how the incorporation of neurobiological principles can drive advances in ML.

• “The theoretical … end of section 5”. Answer: Considering the cases of $\mathbf{W}_r = \mathbf{I}$ and the two-dimensional model, we explore two different limits of arbitrary $\mathbf{W}_r$. The first limit involves relaxing constraints on all parameters in a high-dimensional model while assuming $\mathbf{W}_r = \mathbf{I}$. The second limit involves relaxing constraints on all parameters, including $w_r$, for a 2D model. These two limits provide a foundation for understanding the empirical results regarding stability in models with arbitrary recurrence, which is intractable with the approach used in the paper. There is also a rich history of studying neural mean field theories (Wilson, H. R., & Cowan, J. D. (1972); Kraynyukova, N., & Tchumatchenko, T. (2018)) which yield two-dimensional E-I circuits used for modeling the cortex. The conjecture is crucially important because we want to learn an arbitrary recurrent weight matrix during ML and neuroscience tasks. We have empirical evidence in support of this conjecture (see Fig.4,5 and Fig.1,2 of the attached pdf). We also use this conjecture in the paper itself (for the static input task).

• “The experiments … SSN comparison)”. Answer: We thank the reviewer for the suggestions regarding additional tasks. To satisfy the reviewer, in a subsequent version of the paper, we will include ORGaNICs’ performance on the parity task for comparison. However, we stress that the primary focus of this paper is to demonstrate the stability, trainability, and interpretability of ORGaNICs compared to other RNNs. There was no hyperparameter tuning for any of the tasks – ORGaNICs simply outperforms SSNs on the first attempt at the task we considered.

• “The abstract claims… to be true”. Answer: By that statement, we mean to convey that ORGaNICs mitigates the problem of exploding gradients: through its inherent stability; and vanishing gradients: by processing information across various timescales while maintaining stability, resulting in a blend of lossy and non-lossy neurons. This leads to a trainable RNN circuit, with a performance comparable to LSTMs without the need for specialized techniques like gradient clipping/scaling. We have revised this sentence for better clarity.

• “In this respect, ... training performance”. Answer: Like SSN, ORGaNICs is a neurodynamical model originally designed to explain cortical activity. The comparison with SSN is simply meant to demonstrate that just like SSN, ORGaNICs can be successfully trained on a static MNIST task. So we are not trying to make the point that ORGaNICs performs better due to its stability guarantees, but rather that ORGaNICs can be trained by naive BPTT and perform comparably to SSN trained by the non-trivial method of dynamics-neutral growth (DNG). The result of the comparison is that ORGaNICs performs slightly better (despite no hyperparameter tuning) while being much easier to train than SSN. If we were to speculate on the performance gap between ORGaNICs and SSN, our best guess would be that DNG constrains the range of possible models (viz. parameters) thus reducing the expressivity of SSN. It is beyond the scope of this paper to identify why SSN trained by DNG does not do as well, but what is certain is that it is much harder to train SSN than ORGaNICs. The main takeaway is that ORGaNICs can be trained naively because it is stable, and it performs as well or better than alternative neurodynamical models (e.g., SSN) trained by sophisticated techniques. Regarding the LSTMs, they were trained using gradient clipping in Arjovsky, M., Shah, A., & Bengio, Y. (2016).

• “On clarity of exposition: … navigate”. Answer: We thank the reviewer for the suggestion. We will revise the introduction of the paper to provide a clearer roadmap of the main results of the paper and how they are organized.

Questions:

• “The abstract … backpropagation?” Answer: We mean to convey the fact that we train ORGaNICs by naive backpropagation (not simple gradient descent), meaning that 1) we do not use gradient clipping/scaling that is typically adopted during BPTT when training RNNs such as LSTMs; 2) we do not need to resort to specialized techniques such as the sophisticated DNG method used to train SNN, an alternative neurodynamical model. We have resolved this ambiguity by revising the statement.

• “The formatting … text”. Answer: We thank the reviewer for the suggestion, we have updated the references using the suggested format.

Weaknesses:

• “Generalization: The paper ... the tested benchmarks”. Answer: We expect ORGaNICs to perform well on other ML benchmarks, especially those concerning sequential data. This will be done in a future study, therefore we refrain from making any specific claims about performance on other benchmarks. In this paper, we sought to demonstrate the stability, trainability, and interpretability of ORGaNICs compared to other RNNs, and present proofs and evidence supporting these claims. Specifically, we were able to rigorously prove for the two limiting cases that ORGaNICs is absolutely stable and demonstrated empirically that this stability holds broadly. We achieved good performance on ML tasks using naive BPTT without resorting to gradient clipping/scaling or other techniques required for alternative neurobiological models like SSN, and without hyperparameter tuning. We derived a Lyapunov function for our model offering an interpretable optimization principle explaining the dynamics of ORGaNICs.

• “Performance on Benchmarks: … machine learning models. Answer: There was no hyperparameter tuning nor an attempt to get SOTA performance in this paper. The top-performing ML models are designed with properties of low-dissipation and stability to solve the problem of vanishing and exploding gradients. In contrast, ORGaNICs is a biologically plausible circuit designed to implement divisive normalization (DN), a computational principle experimentally found in a wide range of cortical areas (i.e., different brain regions) across different species, to simulate cortical activity. Unlike ML models, stability in ORGaNICs is not engineered but it emerges from its neurobiological plausible design. Despite not being optimized for the tasks considered, ORGaNICs performs competitively with other RNN models. To our knowledge, no other neurodynamical biologically plausible model achieves this.

• “Scalability: parameter size scales … way for impovement”. Answer: In future work, we will design the gains $\mathbf{b}$ and $\mathbf{b_0}$ to be dependent locally on $\mathbf{y}$ and $\mathbf{a}$, This will significantly reduce the number of parameters compared to the current model where the matrices $\mathbf{W_{by}}$, $\mathbf{W_{ba}}$, $\mathbf{W_{b_0 y}}$ and $\mathbf{W_{b_0 a}}$ are $n \times n$.

Questions:

• “Can this model provide … machine learning models?” Answer: We draw a direct connection between ORGaNICs and systems of coupled damped harmonic oscillators, which have been studied in mechanics and control theory for decades. This connection allows us to derive an interpretable energy function for a high-dimensional ORGaNICs circuit (Eq. 128), providing a normative principle of what the circuit aims to accomplish (see Eq. 13 and the subsequent paragraph). For a relevant ML task, having analytical expressions for the energy function (which is minimized by the dynamics of ORGaNICs) allows us to quantify the relative contributions of the individual neurons in the trained model, offering more interpretability than other RNN architectures. For instance, Eq. 128 reveals that the ratio $\tau_y / \tau_a$ of a neuron pair ('y' and its corresponding 'a') indicates the "weight" a neuron assigns to normalization relative to aligning its responses to the input. This provides a clear functional role for neurons in the trained model. Furthermore, since ORGaNICs is biologically plausible, we understand how the different terms of the dynamical system may be computed biologically within a neural circuit (Heeger & Mackey, 2019). This bridges the gap between theoretical models and biological implementation, providing a framework to test hypotheses about neural computation in real biological systems.

• “Many existing … implemented in this model”. Answer: Divisive normalization (DN) was introduced as a model of the steady-state response of neurons, functioning as a static nonlinearity similar to batch and layer normalization. The DN equation has also been shown to generalize batch and layer normalization, as detailed in the work by Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016). However, in the brain, there are no static nonlinearities; it has thus been proposed that DN is achieved via a recurrent circuit. ORGaNICs is such a recurrent circuit designed so that the responses of neurons at steady state follow the DN equation. While batch and layer normalization are ad hoc implementations that do not affect the dynamics (they are applied to the output layer), ORGaNICs implements DN dynamically. Additionally, whereas batch and layer normalization do not inherently affect the stability of the model (because they do not influence the dynamics), our paper demonstrates that a model implementing DN naturally exhibits stability, which is greatly advantageous for trainability. This stability, derived from the dynamic implementation of DN, sets ORGaNICs apart by providing both output normalization and model robustness.

• “In comparison… limited discussion on this topic”. Answer: We thank the reviewer for this suggestion. We will add a paragraph based on our answer to the first question.

Limitations:

• “In the sequence modeling … the parity task”. Answer: We thank the reviewer for the suggestions regarding additional tasks. To satisfy the reviewer, in a subsequent version of the paper, we will include ORGaNICs’ performance on the parity task. However, we stress that the primary focus of this paper is to demonstrate the stability, trainability, and interpretability of ORGaNICs compared to other RNNs, which we have achieved through a thorough analytical and numerical analysis, and not to provide an extensive benchmark of the model on ML tasks. A more comprehensive test of ORGaNICs on additional long sequence modeling tasks, along with hyperparameter optimization, will be pursued in future work.

Weaknesses:

• “The main … raise my score”. Answer: Our proof for the stability of the high-dimensional model (using the indirect method of Lyapunov) relies on the existence of analytical expressions for the fixed point. Such an expression exists when $\mathbf{W}_r=\mathbf{I}$, but does not exist when $\mathbf{W}_r \neq \mathbf{I}$. However, we were able to prove unconditional stability for the 2-dimensional model (see Fig.1, Theorem.5.1 and Theorem.5.2), for any positive value of $\alpha (w_r) \in (0,\infty)$ and any combination of choices for other parameters. Building on these two limiting cases, for which a rigorous proof of stability is tractable (i.e., for any $w_r$ in mean field, when d=2, and for $\mathbf{W}_r=\mathbf{I}$ when d>2, with no restrictions on other parameters), we conjecture at the end of Section 5 that the stability property holds in higher dimensions for systems where the largest singular value of $\mathbf{W}_r$​ is less than 1 (therefore covering the case when $\alpha \in [0, 1]$). We provide empirical evidence in support of this conjecture in Fig. 4 and Fig. 5. To satisfy the reviewer’s question directly, we simulated 10,000 random networks with $\mathbf{W}_r=\alpha \mathbf{I}$ with $\alpha \in [0,1]$ and plot the distribution of the maximum real part of eigenvalues, all of which are found to be less than 0 (Fig.1 of pdf), indicating stability. We do not exclude that a statistical approach could be used to prove the stability of a general network “in expectation”, but 1) it would fall short of a rigorous proof of stability like ours; 2) it is sufficiently ambitious that it should be the subject of a subsequent paper.

• “Also, the … amount of time”. Answer: We would like to point out to the reviewer an additional experiment at the end of Section 5 where we mention empirical evidence of stability for a more general case of the recurrent matrix. where the maximum singular value of $\mathbf{W}_r$ is constrained to 1. This conjecture is supported by empirical evidence showing consistent stability, as ORGaNICs initialized with random parameters and inputs under these constraints have exhibited stability in 100% of trials Fig.4.” In Fig. 4 the “true” fixed point was found by simulating the network from a zero initial condition. In all of the trials where the conditions in the conjecture were met, the dynamics always converged to the stable fixed point found by the iterative scheme. To show that in all simulations there is no limit cycle and that we can start from a random initial condition, we re-performed this analysis (10,000 networks), starting from random initial conditions \in [0, 1] and fixed $\mathbf{W}_r=\alpha \mathbf{I}$ for some $\alpha \in [0,1]$. We find that the system is stable (Fig.1 of pdf) and the simulations always converge to the same fixed point as found by the iterative scheme. So, while we cannot prove the conjecture by the indirect method of Lyapunov, numerical evidence suggests that the conjecture is true almost surely.

• “For the … and not trained”. Answer: This is an excellent point. We chose the time constants to be learnable to demonstrate that ORGaNICs is competitive with LSTMs on sequential MNIST tasks without hyperparameter optimization and without using ad hoc techniques for training, such as gradient clipping/scaling. As per the referee's suggestion, we have now conducted additional experiments where ORGaNICs is trained with fixed time constants (Table 1 of pdf). Our findings indicate that even with fixed time constants, ORGaNICs maintain stable training without the need for gradient clipping/scaling.

• “The static and … ORGaNICs dynamics”. Answer: We have provided empirical evidence for unconditional stability for a general recurrent weight matrix, particularly when the maximum singular value of $\mathbf{W}_r$ ​ is less than 1 (Section 5, Fig. 4, and Fig. 5). We prove unconditional stability, using a non-trivial approach, in two specific cases of $\mathbf{W}_r$ ​: first, when $\mathbf{W}_r = \mathbf{I}$, and second, for the 2D model. While a proof for unconditional stability for a general $\mathbf{W}_r$​ would be most desirable, it is currently beyond reach as detailed in our response above. Therefore, we must rely on empirical evidence for the stability of a general $\mathbf{W}_r$. In a subsequent version of the paper, we will provide results for the formal language parity task to further corroborate our point.

Questions:

• “What hyperparameters … tasks?” Answer: Learning rate = 0.001 (static MNIST), 0.01 (sMNIST); Batch size = 256, Weight decay = $10^{-5}$ for both. This information has been added in the Appendix.

• “While the … LSTM?” Answer: We do not have a specific scheme for selecting an optimal learning rate. Since the point of this paper is to showcase greater stability, trainability, and interpretability compared to other RNNs, and not to achieve SOTA on the benchmarks, we did not tune the hyperparameters. This will be explored in future work and should lead to better performance on ML benchmarks.

• “In the verification … algorithm?” Answer: The “true” fixed point is found by simulating the network from a zero initial condition. However, in response to the reviewer, we have verified that for a random initialization as well, the simulation converges to the same fixed point.

Limitations:

• “Even for … phenomena”. Answer: This class of models has already been shown to produce non-static oscillatory activity as well as to reproduce a wide range of neural phenomena, including sustained activity, sequential activity, motor preparation and motor control (Heeger, D. J., & Mackey, W. E. (2019)); simulate the dynamics of V1 activity (Heeger, D. J., & Zemlianova, K. O. (2020), S. Rawat, D.J. Heeger, and S. Martiniani. Cosyne Abstracts 2024); predict the emergence of communication subspaces in interareal communication (S. Rawat, D.J. Heeger, and S. Martiniani. Cosyne Abstracts 2023).