Thank you for your insightful review. Your summary, in particular, was helpful in understanding whether our work was sufficiently well communicated.

Re: difficulty in reasoning about the model owing to its moving parts.

This is a very useful comment, thank you. To this end we ran an ablation study on a simplified maze task in order to elucidate any performance differences when ablating NLMs and synchronization. We ran this ablation study on a 15x15 maze task, using 100K training steps and parameter-matched (+-9M) models that we will add in a new appendix:

Referring specifically to the question of the role of synchronization, it serves three important functions:

1. It decouples the representation for downstream use (i.e., attention and projections to predictions) from the raw neuron states ($z^t$), which enables neurons to develop and enhance dynamics without being constrained by the immediate expectations of the representation itself. In other words, the synchronization representation lets a rich representation unfold and stabilize while still allowing the neurons themselves to oscillate and develop patterns over long time scales.
2. It enables multi-scale temporal sensitivity, by way of the learnable decay parameters associated with neuron pairs ($r_{ij}$).
3. It enables higher cardinality representations since the number of neuron pairs is far greater than the number of neurons. Using pairwise decay parameters unique for each synchronization representation (i.e., there are multiple, and can be many multiple in the future), this further differentiates the functionality of each pairing in time, not just space.

We explored several techniques for extracting usable representations from the latent space of the CTM ($z^t$), including but not limited to: direct linear projections therefrom, MLP-based projections therefrom, gated projections, all of the aforementioned but into some ‘holder’ representation that updated slowly, and even 1D convolutions. None of these offered the same level of simplicity, flexibility, stability, and representational capacity that synchronization representations could. As soon as we started using synchronization we observed an ‘unlocking’ to the complexity of the neural dynamics that we had yet to see to such a degree.

---

Re: questions

Re: language model.

We agree with that sentiment. However, the space of language modelling is extremely competitive and not necessarily well-suited as a test-bed for a truly novel modelling approach. The CTM can certainly benefit from the community effort required to transition a new model to a truly competitive one, but we simply do not have the resources to do that on our own. Hence, we focused on interesting experiments that could yield observations and findings that might eventually serve to understand artificial intelligence on a deeper level.

Re: why the best tick in the loss function?

This is an insightful question. You are right in thinking that it is not strictly necessary. The effect of including the best tick is that it promotes certainty in that it drives the best prediction to be even better. We found that this helped bootstrap learning, although it could benefit from further exploration in the future.

Re: Emergent phenomena.

There simply is no additional space in the main text to flesh these out. Further, many of these observations were as a consequence of the “journey” taken to develop this work, and would be difficult to re-explore after having learnt the lessons the observations had on offer. That said, we agree that future experiments into some of these phenomena would be fascinating and are currently investigating related concepts.

---

We hope that this response addresses your outstanding concerns and answers your questions. It would be extremely helpful to engage with you further, particularly if there is anything you think we could do that would prompt you to consider improving your review score.

Thank you for your insightful and directed review. We appreciate the important points that you have raised regarding novelty and prior work.

Re: Clarifying novelty

We agree that our novelty claim, regarding synchronization, was insufficiently posited within these prior works. This lack was an honest oversight, but the specific details of what makes our use of synchronization novel still stand. Specifically, and crucially, we use synchronization as learnable representations. These representations are used downstream via weighted projections (e.g., for attention queries and class predictions, although many such representations can be extracted from the singular dynamical system that the CTM defines).

Reichert & Serre (2014) introduced a model where synchronization emerges from the interaction of complex-valued neurons, where the mechanism is the phased interaction of complex weighted-sums of inputs, and the role of their synchronization is as a modulator of information flow, effectively binding neurons in a form of gating. They make use of these emerging ‘bound’ neuron groups for tasks such as object segmentation, using a post-hoc processing step to use the synchronized neurons. While interesting and useful, this is quite distinct from our direct use of synchronization as a latent representation.

In the context of complex-valued neural networks, there are a variety of different architectures that utilize neural timing and synchronization (Lee et al. 2022). These use “synchronization” in the traditional control and dynamical systems sense: they develop controllers or conditions to make complex-valued neural network models synchronize with each other or reach stable synchronized states. By contrast, the CTM uses neural synchronization in a computation sense: it treats the synchronization of neuron activities as a feature representation within a neural network model. There is no external “controller” in CTM enforcing synchronization; instead, CTM’s training optimizes how neurons’ activation patterns synchronize so that this synchrony encodes useful information for tasks. Essentially, CTM repurposes the concept of synchrony as a tool for thinking, whereas the other works treat synchrony as a behavior to be achieved for system stability or control objectives. We believe this makes the CTM fundamentally different from those prior synchronization studies in how the concept is applied.

Actions:

• We will completely clarify that our novel use of synchronization is as a latent, learnable, and internally constructed representation that the model builds and uses downstream. To that end, we propose to rephrase the paragraph starting on line 46 as:

  “The CTM learns to use neural synchronization as its latent representation, distinguishing it from existing work that explores synchrony as emergent properties for post-hoc use [citations, including Reichert and Serre]. This representation is distinct from the common static `snapshot' representations used in most modern NNs as it directly encodes the temporal interplay of neural dynamics.”

• Regarding line 65: this was perhaps a poor phrasing of what we intended to say, as we did not mean to claim that the CTM is the introduction to the use of neural timing, but rather that it uses timing and synchronization as computational tools for its learning. Thank you for pointing that out. We will also rephrase Line 65 to read as:

  “The CTM uses neural timing and synchronization as core computational principles...”

• We propose to include the following paragraph (subject to minor tweaks due to space constraints) in the related work section, dedicated to earlier explorations in synchrony (please feel free to offer critique and/or make requests):

  “Reichert & Serre (2014) proposed a model where synchronization emerges from interactions among complex-valued neurons, serving as a gating mechanism that modulates information flow and enables post-hoc grouping of neurons for tasks like object segmentation. Unlike CTM, however, their model does not use synchrony as a learned latent representation during computation. Other approaches in complex-valued neural networks (Lee et al., 2022) employ synchronization from a control-theoretic perspective, aiming to stabilize or coordinate networks via externally enforced synchrony. In contrast, CTM integrates synchronization intrinsically, optimizing neural phase relationships during training to encode task-relevant representations. This positions CTM as a computationally grounded model of synchrony, fundamentally distinct from prior works that treat synchrony as a control objective.”

---

Re: abstracting away dynamics

Thank you for pointing out this important distinction. We do, indeed, mean single-neuron models, hence the introduction of NLMs, enabling a more complex (learnable) neuron model. We propose to clarify this in the abstract, replacing the sentence you pointed out with the following:

“Most artificial neural networks ignore the complexity of individual neurons.”

---

Additional note: Ablations

In response to other reviewers we have performed an ablation study to elucidate the impact of NLMs and synchronization. We ran this ablation study on a 15x15 maze task, using 100K training steps and parameter-matched (+-9M) models that we will add in a new appendix:

---

We hope that this sufficiently clarifies our claims and better positions our paper in the full breadth of prior art. Thank you for your excellent suggestions and nuanced questions. Please let us know if we can do anything more, or if you have any further questions.

We sincerely thank you for your thoughtful feedback and are encouraged by the positive assessment of our work. The questions raised are insightful and touch upon some of our core design principles. We hope that addressing these points will strengthen the paper.

Weaknesses and key questions

You have highlighted two primary concerns: (1) the CTM's sensitivity to initialization in the parity task and (2) the apparent tension between long training times and biological plausibility. We will address these crucial points first.

1. Training stability/sensitivity to seed

You are correct that different random seeds can lead to varied final performance. We discuss and analyze this in Appendix F.6.

Crucially, we argue that this variance is not owing to arbitrary instability but rather a direct and fascinating consequence of the CTM discovering qualitatively different algorithmic strategies to solve the task. Our analysis in Figure 20 shows that different runs converge to distinct, interpretable algorithms:

• One successful run learns a forward-pass strategy, scanning the sequence from start to end.
• Another successful run learns a reverse-pass strategy, attending to the sequence from end to start.
• Sub-optimal runs often learn incomplete or less efficient hybrid strategies.

This emergent property actually highlights a strength of the CTM: its representational space, built on neural synchronization, is rich enough to enable the discovery of distinct computational solutions. While this can lead to variance in performance depending on the “quality” of the discovered algorithm, it also makes the CTM a powerful tool for investigating how models learn to solve procedural tasks, not just if they can. The interpretability benefits of the CTM are noteworthy in this case. In the future, we hope to develop techniques for driving towards and/or selecting from the distinct computational solutions the CTM can learn.

To ensure this point is not missed, we will add a pointer to the main text in Section 6.1 that explicitly references the detailed analysis of these emergent strategies in Appendix F.6.

---

2. Biological plausibility and training times

You have made an insightful, and also deeply philosophical point. The comparison between training in deep learning and learning in biology begets a careful consideration of timescales. Consider the following:

• Evolutionary and developmental “training”: The priming of biological brains has occurred over millions of years of evolution, shaping architecture, learning rules, etc. This is followed by years of childhood learning. Taken in totality, such a protracted process is somewhat analogous to the use of millions of gradient descent iterations to train a model’s internal machinery from scratch.
• In-situ, rapid learning: The remarkable few-shot or one-shot learning that humans exhibit (e.g., learning a new object class) is possible only because it builds upon this vast foundation of prior “training”.

Our work aims to capture some functional dynamics of an “already trained” brain, rather than simulating its entire evolutionary and developmental trajectory, hence the long training times. Furthermore, a key feature of the CTM is its native adaptive compute (Section 5.1). This principle directly aligns with that of biological efficiency at inference time. The CTM naturally learns to “think longer” for challenging inputs, which is to say biologically-plausible ‘rapid’ timescale intelligent resource allocation.

---

3. Other questions

Re: NLM parameter count

Yes, they do. We acknowledge this trade-off directly in our limitations paragraph (Section 8, line 277).

We view this not as a simple drawback, but as a justified design choice that provides a new axis for model scaling and expressivity. Per-neuron MLPs is what enables each neuron to learn a unique, complex internal model, which in turn is fundamental to generating the neural dynamics at the core of the CTM.

Re: Neuron model types

This is an excellent query of our modeling philosophy. Our approach is one of biological inspiration, not emulation. We briefly discuss this choice in the conclusion on line 271, and will expand on this as per another reviewer's request.

Models like that of Hodgkin-Huxley are architecturally more realistic representations of the underlying functionality of neurons. These types of neuron models are described by complex, non-linear differential equations. Integrating these models within large-scale, gradient-based deep learning frameworks is, with current methods, computationally prohibitive and often non-differentiable. Taking this approach would have drastically limited the types of experiments we could undertake and would most likely not have yielded the observations we share in this paper.

Instead, we chose to abstract the functional role of a neuron in a dynamic system: to produce a complex temporal output based on its history of inputs. A simple MLP enables the CTM to learn the optimal dynamic response for the task at hand. Crucially, this choice ensures that learning remains tractable, end-to-end differentiable, and compatible with current deep learning paradigms. Our intentional choice in this matter is what allows us to bridge principles (not emulations) from neuroscience with findings and techniques of modern AI.

---

Additional note: Ablations

In response to other reviewers we have performed an ablation study to elucidate the impact of NLMs and synchronization. We ran this ablation study on a 15x15 maze task, using 100K training steps and parameter-matched (+-9M) models that we will add in a new appendix:

---

We hope these responses have clarified our design choices and the novel implications of our work. Please feel free to ask any additional questions or propose changes. Thank you.

Thank you for your in-depth review and comments. It is clear that you unpacked and understood both the motivation and efforts of our work.

---

Regarding weaknesses

Clarity issues.

1. We have removed the word ‘teleport’. Its use was to capture the process of re-applying the same model sequentially by shifting the start location to the predicted (valid) end location (determined by following the steps predicted by the model). We propose replacing the end of the caption with:

‘... (c) generalizing to 99 x 99 via sequential re-applications of the same model.’

2. Re Figures 1 and 2.
   We will contextualize those figures in the second contribution, providing pointers appropriately. We apologize for this oversight.

3. Re Figure 3.
   We appreciate this feedback. Unfortunately, visualizing a time-based architecture in a static figure is inherently difficult. This is why we created the supplementary animated video ‘arch.mp4’. In our revision, we will explicitly reference this video in the caption, framing it as an essential companion for understanding the model's operational flow.

4. Re the use of the word “synchronization” instead of “temporal correlation”.
   We chose “synchronization” because it describes the emergent, coordinated functional behavior we aim to model, a biologically-inspired concept central to population coding in neuroscience. “Temporal correlation”, on the other hand, is the underlying mathematical mechanism we use to implement it. To make this point clear, we propose the following change when first introducing the term (in the contributions):

“...the use of neural synchronization directly as the representation (implemented via temporal correlations between neuron-level activity; Section 3.4)…”

5. Re terminology, particularly referring to “synchronization as a representation”.
   We explicitly define the meaning of synchronization as a representation using equations in Section 3.4. We now see the need for a slightly more exhaustive descriptive introduction - the above response addresses this.

---

Regarding quantitative rigor:

Thank you for pushing for greater rigor. To that end, we have taken two direct actions:

1. We have run ablations (details to follow).
2. We will clarify that our baselines are already parameter-matched where applicable.

With that said, we would also like to share the perspective that guided our experimental design. A traditional approach is to immediately position a new idea against SOTA benchmarks. However, we are cognizant that today's leading models are the product of years of compounding engineering gains and fine-tuning on those specific tasks. Given this, we believe that for a fundamentally new architecture like the CTM, a premature focus on SOTA performance would distract from this paper's core scientific contribution: understanding what the CTM does and how its temporal mechanisms operate.

Our primary goal, therefore, was to establish the what and how of the CTM's unique operation first, providing a stronger foundation for the community. This required designing novel experiments to specifically probe its capabilities. For example, finding the right “hello world” task was a challenge in itself; our phrasing of the maze task - which requires forming an internal world model by disallowing positional embeddings and outputting a direct action sequence - was a deliberate choice to truly showcase the nuanced differences between the CTM and recurrent baselines like LSTMs. We believe this setup reveals more about our architecture's potential for complex reasoning than a raw performance number on a standard classification benchmark might.

We understand your desire for more rigorous quantitative metrics and hope this response clarifies our rationale and strengthens our contribution.

---

Regarding biological grounding:

You have raised an important point about the level of abstraction. Our approach is one of biological inspiration, not simulation. The NLM is a functional abstraction of the complex, time-dependent integration that occurs in biological neurons, rather than a literal model of a specific subcellular mechanism. We use a truncated FIFO history (as noted in line 116) as a computationally tractable way to provide this temporal context. We propose the following change to the first paragraph of line 271:

“The CTM's NLMs are inspired by the complexity of biological neurons, but are implemented with a level of abstraction appropriate for modern deep learning.”

---

Regarding limited ablations:

We have run an ablation study on a 15x15 maze task, using 100K training steps and parameter-matched (+-9M) models that we will add in a new appendix. The results are as follows:

---

Regarding questions

Synchronization representation.

Thank you for asking for this clarification. Section 3.4 defines synchronization mathematically as the temporally-decayed correlation of neuron activity histories. This representation is crucial for three reasons:

1. Decoupling: It decouples the representation used for downstream tasks from the raw neuron states ($z_t$), allowing the neurons to develop richer internal dynamics without being solely constrained by the immediate prediction loss.
2. Hebbian Inspiration: It captures the idea of “neurons that fire together, wire together.” By making the correlation of activity the representation, we directly model the interplay within the neural population over time.
3. Multi-scale Temporal Sensitivity: The learnable decay parameter ($r_{ij}$) for each neuron pair allows the model to learn a representation that is sensitive to phenomena at different timescales simultaneously, which is critical for complex sequential reasoning.

Owing to the lack of space in the main paper, expanding on these points in the main text is not possible. That being said, we can potentially write a new appendix that goes into more detail about the nature of the synchronization representation, should the reviewer request that.

---

Decaying influence kernel.

Thank you for this question. The decaying influence kernel gives more weight to recent ticks. As defined in Equation 9, the influence of a past tick, $\tau$, on the current tick, $t$, is scaled by $\exp(-r_{ij}(t-\tau))$. Because the difference $(t-\tau)$ is large for distant ticks and small for recent ones, the negative exponent correctly implements an exponential decay of influence over time.

For example, for $t=3$, the numerator of Equation 10 expands to $z_i(1)z_j(1)\exp(-2r) + z_i(2)z_j(2)\exp(-r) + z_i(3)z_j(3)\exp(0)$ which correctly accounts for a larger influence of the current tick ($z_i(3)z_j(3)$) compared to the previous ticks.

---

Biological plausibility.

As discussed in our response to the weaknesses, our goal is biological inspiration, not direct simulation. The CTM’s NLMs are an abstraction that enables learning more complexity on a neuron-level, as opposed to an attempt at emulation. Please see the clarification and proposed adjustment to the text (in the weaknesses section, above).

---

Subsampling neuron pairs.

• Sensitivity: The model is not highly sensitive to the specific random sample of pairs, provided the number of pairs ($D_{out}$, $D_{action}$) is sufficiently large for the task's complexity. For our experiments, we generally chose these values to be on the same order of magnitude as the model's width or the task's output dimensionality.
• Resampling: The pairs are sampled once at initialization and then fixed. Resampling at each tick is not feasible, as the projection weights ($W_{out}$, $W_{in}$) are learned specifically for the chosen pairs. We will clarify this in Section 3.4.1.

---

Ablations.

As mentioned in our response to weaknesses, we have now completed an ablation study that shows NLMs and synchronization are both beneficial.

---

“Looking around” rationale.

This is a fantastic question about the functional role of this emergent behavior. When we built the CTM there was never any explicit objective for “looking around”, so when we observed this behaviour we were struck by its parallels to how we expect an intelligent agent to observe a scene. Thus, we believe its emergence points to both a compelling parallel with observable intelligence and a tangible computational benefit.

From a conceptual standpoint, we often identify intelligence functionally through action. A crow solving a puzzle is deemed intelligent because it undertakes actions to achieve a goal. Similarly, the CTM's emergent “gaze” behaviour is an active process of information gathering. That this behavior arises without any direct supervision is a fascinating result because it parallels an intuitive aspect of how intelligent agents interact with their environment.

More crucially, from a computational perspective, this “looking around” process is the mechanism that enables native adaptive computation. By iteratively attending to different parts of the image over its internal ticks, the CTM can refine its understanding and adjust its processing time based on the input's complexity. As shown in Figure 5a, this allows it to spend more “thought” (more internal ticks) on difficult images compared to simpler ones. This input-dependent allocation of compute is a direct, beneficial consequence of the architecture's temporal design.

We have added clarification to Section 5.2. as follows:

“...It does this entirely without prompting or any guide, implementing computationally beneficial adaptive compute in an intuitive fashion…”

---

Thank you again for your thorough review. Please feel free to ask more questions.