Large Language Models as Model Organisms for Human Associative Learning

We thank the reviewer for the insightful feedback.

Summary of main points:

• Q1: we clarify Fig. 2a and 2b and show that mid-similarity pair differentiation remains statistically significant after correction for multiple comparisons. While [28] does not provide quantitative predictions, our results are consistent with its core theoretical claims, even though we agree that integration effects for high-similarity pairs are weaker.
• Q3: we clarify Fig 3 and our proposed vocabulary interference measure as a plausible factor influencing representational dynamics. This interpretation remains exploratory, as global stimulus interference has not, to our knowledge, been systematically studied in neuroscience studies. Estimating such effects in humans would require modeling an individual’s lifetime exposure to concepts, a challenging task. Nonetheless, by leveraging the fully observable space of LLMs, we are able to investigate this underexplored factor, grounding our interpretation in neuroscience principles.
• Q4: we tested WordNet-based token pairs and found that high vocabulary interference led to a shift from U-shaped to monotonic representational change. This supports our claim in Sec 3 that NMPH predictions depend on global similarity structure.

Q1 [Fig 2b & significance of NMPH]: In Fig 2b we aggregate the representational change across models, similarity groups, and repetition counts that correspond to the consolidation phase (as defined per model in Tab 3 Appendix C). We will further include per-model and per-repetition plots in the appendix so readers can assess the consistency and robustness of the U-shape across settings across conditions.

The work that proposed the NMPH [28] provides a theoretical account of how representational change should vary with memory coactivation, but it does not specify quantitative predictions, making it difficult to define how pronounced a U-shape must be to support the hypothesis. For empirical context, [39] report a differentiation effect of approximately -0.1 (Pearson correlation) for mid-similarity pairs in human fMRI data. While this offers a useful benchmark, our analysis uses cosine similarity and is conducted on LLM representations–direct comparisons of effect size are not possible. Nevertheless, the ~0.15 change we observe is robust and statistically significant across all 6 models (after multiple comparisons correction; see reply to reviewer ctgT). While we do not observe clear integration for high-similarity pairs, a factor that contributes to the U-shape being less pronounced, the differentiation in the mid-similarity range remains consistent with core predictions of the NMPH, and we believe this provides meaningful support for the hypothesis as illustrated in Fig. 2b.

Q2 [Fig 2a]: We normalize the x-axis to allow alignment of different models based on their phase transitions (from encoding to consolidation to forgetting). Each model enters and exits these phases at different repetition counts due to a slight variability in their learning dynamics (as can be observed in Tab. 3 of Appendix C). To enable visual comparison across models, we map the repetitions for each phase into fixed intervals on the x-axis: (0,1) for encoding, (1,2) for consolidation, and (2,3) for forgetting. Within each model, the actual repetition counts in a phase are linearly scaled to fit within these ranges. This approach ensures that phase-aligned trends are visible despite underlying differences in repetition scale. We will revise the figure caption and text to more clearly state this normalization and add the original accuracy per model in the Appendix. Please let us know if further clarification is necessary.

Q3 [Fig 3]: We agree that the interpretation of Fig 3b involves a degree of speculation. This is, in part, due to the fact that, global stimulus interference has not been systematically studied in memory neuroscience, to the best of our knowledge. Estimating such global effects in humans would require modeling an individual’s prior exposure to all words and concepts, which is extremely challenging and remains an open problem in cognitive neuroscience. For example, [1] explicitly discusses the difficulty of estimating and accounting for the impact of priors in human memory studies. By leveraging the fully observable representational space of LLMs, we are able to explore this under-investigated factor and propose vocabulary interference as a potential mechanism influencing the different and seemingly contradictory findings in humans. While our interpretation is necessarily exploratory, we believe it offers a plausible explanation grounded in principles from neuroscience – specifically, that memory systems must balance the competing demands of integration and interference resolution. We will revise the main text to (1) make the speculative nature of the explanation more explicit, and (2) more clearly connect it to the underlying theory and the observed behavior in Fig. 3(b).

Q4 [Semantically meaningful tokens]: Our study intentionally used token pairs selected for their pre-learning similarity, regardless of semantic meaning. This approach is similar to the use of synthetic stimuli in [39], which highlights the need to sample across the full similarity spectrum (especially the mid-similarity range) to test the NMPH effectively. Real-word tokens are unevenly distributed across this space, making precise control difficult. Our main goal in this paper is not to study meaning, but the structural dynamics of representational change in response to learning. However, we do agree that using meaningful tokens is an important analysis to understand how the NMPH applies in real-world learning. We sampled new token pairs, restricting the search space to single-token words from WordNet. This filtering step reduced the usable vocabulary to approximately 20% of the original token set in the two models tested by the time of this reply.

We first examined how pairwise similarity and vocabulary interference were distributed within this space, anticipating that the limited vocabulary might bias the pairs toward a narrower range of interference values. We found that real-word token similarity more closely reproduces the vocabulary interference distribution. According to our framework in Sec. 3, this implies that most of these pairs would experience high vocabulary competition during prediction. Based on this observation, we hypothesized that the representational change curves would no longer exhibit a U-shaped trend but instead resemble a linearly decreasing pattern, as seen in the orange and blue curves of Fig. 3b. This prediction was confirmed in our results: post-learning similarity decreased monotonically with pre-learning similarity, supporting the idea that highly similar token pairs undergo representational changes influenced by the degree of vocabulary interference.

We will include this new analysis in the revised paper and revise our claims accordingly. Specifically, we will clarify that while the NMPH provides a useful theoretical framework, its behavioral predictions may depend on the distributional properties of real-world stimuli. Our findings suggest that to fully explain representational change in naturalistic learning contexts, NMPH-like theories may need to account for global similarity structure, such as vocabulary interference, as a modulating factor.

Q5 [h_y]: In our setup, the model is prompted to predict a two-token sequence, where the first predicted token should be $y$ for the prediction to be considered correct. We obtain $h_x$ as the final hidden state of the input tokens, and $h_y$ as the hidden state of the first predicted token $y$ (that is, the first token generated by the model, which is not part of the input). Although $y$ is not explicitly fed into the model, we can still access its contextualized representation through the generation process. To validate the robustness of our method, we also conducted a control experiment in a non-predictive setting, where both $x$ and $y$ were given to the model as input, and $h_y$ was extracted directly from the input sequence. The results, including the U-shaped pattern, were qualitatively similar in both predictive and non-predictive settings. We will revise the main text to clarify this distinction between $h_x$ and $h_y$.

Q6 [Extension to sentences]: We appreciate the reviewer’s question. Extending this work to sentence-level inputs is an interesting direction, but it introduces several challenges. First, it is nontrivial to construct sentence pairs with controlled and quantifiable pre-learning similarity, which is essential for testing nonmonotonic representational change as predicted by the NMPH framework. Second, it is unclear what specific patterns one should expect from sentence-level associations, given the added complexity of syntax and semantics. Our study intentionally focused on a simplified associative learning task using token pairs, in line with the experimental design used in neuroscience (e.g., [Reb1]), to evaluate whether theoretical predictions about hippocampal learning dynamics extend to LLMs. This controlled setup allowed us to isolate and investigate representational changes in a targeted and interpretable way.

[Reb1] Stark, Shauna M., et al. "What’s in a context? Cautions, limitations, and potential paths forward." Neuroscience letters680 (2018): 77-87.

We believe we have addressed the main points and welcome any follow-up questions.

We thank the reviewer for the insightful feedback.

Summary of main points:

• Q1: studying NMPH in humans is challenging due to limited control over pre-learning similarity, task-dependent effects, and constraints like noise and cost. Human studies also struggle to account for global knowledge structure. We argue that LLMs, though biologically distinct, offer a valuable model system for testing representational-level hypotheses under controlled and fully observable conditions.
• Q4: we considered this, but untrained LLMs are unsuitable for our paired-associate task due to the lack of ICL abilities.
• Q5: we used synthetic token pairs selected by pre-learning similarity, independent of semantics, to ensure full coverage of the similarity spectrum, especially the mid-similarity range critical for testing NMPH [39]. Token pair examples per group are now included. We also explored real-word tokens and found their similarity aligns with vocabulary interference, shifting the representational change curve from U-shaped to monotonic. This supports our claim that global interference is a key modulating factor.

Q1 [Foundation of the research]: Studying the NMPH in humans presents several challenges. First, it is extremely difficult to precisely control the pre-learning similarity between associated items – a core requirement for detecting nonmonotonic representational change, as emphasized in [39]. Defining what counts as “0.1” vs. “0.5” similarity in humans is nontrivial, costly, and time-consuming. For this reason, [39] relied on representational inversions of DNNs to generate visual stimuli with controlled perceptual similarity. Second, the similarity level at which differentiation emerges varies by task and stimulus set, making it unclear in advance which mid-similarity range will reveal the effect. This requires dense sampling across the similarity continuum, adding further complexity. Finally, as with any human behavioral experiment, results are constrained by cost, limited trials, participant fatigue, and noise. Beyond pairwise similarity, additional factors (such as global (vocabulary-level) interference) may also influence learning dynamics. Measuring this in humans would require estimating an individual’s lifetime exposure to words and concepts, which is inherently difficult and practically infeasible. As a result, human studies typically do not account for how prior knowledge structure might modulate representational change. There is empirical support for all three hypotheses discussed in our paper [6, 9, 10, 28, 30, 32]. However, these findings are often empirically inconsistent, and it remains unclear when each dynamic arises, under what conditions, and which factors drive the observed differences. We propose that global interference may be one such factor influencing these outcomes. For instance, in memory systems such as the hippocampus, similar representations are thought to compete during retrieval, and the system works to disambiguate overlapping information. [9] note, “The hippocampus is believed to reduce memory interference by disambiguating neural representations of similar events.”. This suggests that similarity beyond just the target pair (i.e., relative to the broader knowledge space) may play an important role in shaping representational dynamics. LLMs offer a tractable tool for testing representational-level hypotheses. Unlike biological systems, they provide full access to internal representations, explicit control over exposure, similarity, and repetition, and complete observability of prior knowledge.This enables precise computation of local and global similarity, systematic testing of learning dynamics, and exploration of how broader structure shapes outcomes. While we do not claim that LLMs are mechanistically equivalent to human brains, we view them as complementary model systems – ideal for generating and testing hypotheses that are difficult or impossible to probe directly in human subjects.

Q2 [Comparison with mature human brains]: We agree that LLM pretraining differs from human developmental learning. Our comparisons focus on mature human brains, aligning with the existing neuroscience literature on representational change. We will revise the paper to make our focus more clear, and note this in the limitations section.

Q3 [Factors influencing results]: We tested 6 LLMs with diverse architectures and pretraining data and observed consistent trends across them, suggesting that the representational changes we study reflect a broader pattern not tied to any one model or dataset.

Q4 [Evaluating untrained LLM]: While testing untrained models is a useful control in some contexts, our task requires models to be able to effectively learn paired associates. This is not possible with untrained LLMs. Still, we agree it would be valuable future work to study how these representational dynamics emerge during training, especially under a curriculum that mimics human developmental learning.

Q5 [Examples of token pairs]: Below we provide randomly selected examples per similarity group, based on optimization for llama2-7b. Similar examples for other models will also be included in the appendix.

Our study intentionally used token pairs selected for their pre-learning similarity, regardless of semantic meaning. This approach is similar to the use of synthetic stimuli in [39], which highlights the need to sample across the full similarity spectrum (especially the mid-similarity range) to test the NMPH effectively. Real-word tokens are unevenly distributed across this space, making precise control difficult. Our main goal in this paper is not to study meaning, but the structural dynamics of representational change in response to learning. We agree that using meaningful tokens is important for assessing how NMPH applies in real-world learning. To explore this, we sampled token pairs restricted to single-token WordNet words, reducing the usable vocabulary to ~20% in the two models tested. We found that real-word token similarity closely aligns with vocabulary interference, implying that most pairs face high competition during prediction. Based on this, we hypothesized – and confirmed – that the representational change curve would shift from a U-shape to a linearly decreasing pattern (similar to orange and blue curves in Fig 3b). We will include this analysis in the revised paper and clarify that while NMPH remains a useful framework, its predictions may depend on the similarity structure of real-world stimuli. Our findings suggest that global interference should be considered as a modulating factor in naturalistic learning settings. See reply to reviewer ctgT for more details and semantically-meaningful token pair examples.

Synthetic token pairs:

We did not use alternative prompt templates, opting for a minimal prompting setup to isolate learning effects and reduce confounds from prompt structure.

Q6 (intermediate layers): We agree this is an interesting point. We examined intermediate layers in an appendix figure, which we later found to have an error. Since NeurIPS does not support figures or PDFs during the rebuttal, we summarize the results below for two different experimental settings:

1. Using token pairs optimized for similarity in earlier layers, (similar to Fig 7a, but fixing a coding error). Our corrected results show:
   • Intermediate and late layers show a monotonic pattern (e.g. similar to Fig 2b, green line), with significant differentiation for high similarity pairs (>0.7 pre-learning similarity.
   • Intermediate layers show stronger differentiation (greater decreases in similarity) than late layers
   • Early layers (1 & 2) behave more erratically, lacking a clear trend
2. Using token pairs optimized for last layer similarity (same tokens as Fig 2b). This analysis is still running, but we will update the rebuttal when our results come in. Other comments:

[Learning phases across LLMs]: All models exhibited encoding and consolidation phases; only two showed a forgetting phase. No models failed to show encoding or consolidation.

[Noise between tokens]: We intentionally avoided adding noise (e.g., distractors or varying context) to isolate learning-related representational changes between specific token pairs. This controlled setup minimizes confounds and aligns with our goal of probing structural learning dynamics rather than robustness to noise.

[Successful and failed LLMS]: If the reviewer refers to whether models exhibit forgetting, we address this in Appendix C.1, which summarizes model-specific behaviors.

We believe we have addressed the main points and welcome any follow-up questions.

We thank the reviewer for the valuable feedback.

Summary of main points:

• Q1: we clarify that our vocabulary interference metric quantifies how densely a token is represented in embedding space. This measure is grounded in cognitive theories of memory interference. We find that higher vocabulary interference is associated with reduced pairwise similarity after learning, across all pre-learning similarity levels. We also find that NMPH-like effects fade under higher vocabulary interference, which may help account for some of the divergent patterns observed in neuroscience studies (differentiation, integration, non-monotonicity).

• Q2: while the NMPH provides a theoretical account of how representational change should vary with memory coactivation, it lacks quantitative predictions—making it difficult to define how pronounced a U-shape must be to support the hypothesis. Our findings nonetheless align with its core idea: we observe statistically significant differentiation for mid-similarity pairs across six models, particularly in the low vocabulary interference range. Although integration is weaker for high-similarity pairs, the overall pattern supports NMPH’s central claims and is consistent with prior empirical work [39].

Q1 [Clarifying vocabulary interference]: We appreciate the reviewer’s comments and the opportunity to clarify. In our paper, we considered the possibility that some of the divergent or controversial findings in the neuroscience literature could be due to a variable difficult to measure in human studies: the global associative structure between stimuli, which in our work we call vocabulary interference. This is intended to measure representational competition from the entire stimulus space – that is, how many other tokens in the vocabulary are highly similar to a given token x. This link between similarity and long-term memory interference is supported by decades of psychology and neuroscience literature [Reb1-Reb5, 9, 23], leading to an influential computational model [23] showing how the hippocampus can resolve interference by disambiguating similar representations. This perspective suggests that representational similarity beyond the immediate pair (i.e., within the model’s broader representational space) could play a key role in modulating learning dynamics, and may help explain variability in empirical outcomes. We operationalize this idea by computing the average cosine similarity between x and all other vocabulary tokens prior to learning. Regarding the conditioning: in our experimental design, we analyze the final $x,y$ pair in a stimulus stream such as $[x_1, y_1, x_2, y_2]$. The representations we use $h(x_2 | x_1, y_1)$ and $h(y_2 | x_1, y_1, x_2)$ are both contextualized within the same prompt. We compute similarity between these two vectors to capture how much the model binds $x$ to $y$ after learning. The vocabulary interference measure is computed before learning, and is not used to define the similarity, but to assess whether regions with higher representational density tend to show weaker learning (i.e., smaller representational change). Importantly, we find that when we control for vocabulary interference, the post-learning similarity curves change systematically: higher vocabulary interference is associated with reduced pairwise similarity after learning, across all pre-learning similarity levels. We will revise the text to clarify the motivation, definition, and limitations of this interference measure. If further concerns remain, we welcome continued discussion as we aim to make the paper as clear and rigorous as possible.

Q2 [Validity of NMPH results]: The work that proposed the NMPH [28] provides a theoretical account of how representational change should vary with memory coactivation, but it does not specify quantitative predictions, making it difficult to define how pronounced a U-shape must be to support the hypothesis. For empirical context, [39] report a differentiation effect of approximately -0.1 (Pearson correlation) for mid-similarity pairs in human fMRI data. While this offers a useful benchmark, our analysis uses cosine similarity and is conducted on LLM representations -- thus, direct comparisons of effect size are not possible. Nevertheless, the ~0.15 change we observe is robust and statistically significant across all 6 models (after multiple comparisons correction; see reply to reviewer ctgT). While we do not observe clear integration for high-similarity pairs, a factor that contributes to the U-shape being less pronounced, the differentiation in the mid-similarity range remains consistent with core predictions of the NMPH, and we believe this provides meaningful support for the hypothesis as illustrated in Fig. 2b.

Q3 [Forgetting phase]: While a detailed analysis of the forgetting phase is beyond the scope of this paper, we provide initial observations and hypotheses in lines (208-215) and Appendix C.1. Specifically, we note that only two models exhibit this behavior. In one case, we suspect it may be related to the model’s use of a sliding window attention (SWA) mechanism, which could limit its ability to integrate information over longer repetition streams. In the other, we hypothesize that high vocabulary interference may play a role in suppressing learned associations. While we do not claim that forgetting is a canonical part of the learning process across all LLMs, we believe these preliminary findings highlight interesting model-specific dynamics that could lead to further investigation in future work.

Limitations: We will add a statement in the limitations section acknowledging that LLMs are not direct stand-ins for humans, and that empirical validation in human studies remains essential.

References:

[Reb1] A. D. Baddeley, “The Influence of Acoustic and Semantic Similarity on Long-term Memory for Word Sequences,” Quarterly Journal of Experimental Psychology, vol. 18, no. 4, pp. 302–309, Nov. 1966.

[Reb2] M. C. Anderson and B. A. Spellman, “On the status of inhibitory mechanisms in cognition: Memory retrieval as a model case,” Psychological Review, vol. 102, no. 1, pp. 68–100, 1995.

[Reb3] M. C. Anderson, C. Green, and K. C. McCulloch, “Similarity and inhibition in long-term memory: Evidence for a two-factor theory,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 26, no. 5, pp. 1141–1159, 2000.

[Reb4] J. B. Caplan, M. Rehani, and J. C. Andrews, “Associations Compete Directly in Memory,” Quarterly Journal of Experimental Psychology, vol. 67, no. 5, pp. 955–978, May 2014.

[Reb5] Y. Zhao, A. J. H. Chanales, and B. A. Kuhl, “Adaptive Memory Distortions Are Predicted by Feature Representations in Parietal Cortex,” J. Neurosci., vol. 41, no. 13, pp. 3014–3024, Mar. 2021.

We hope we have addressed the reviewers' main concerns and we would happily go into more detail if there are any remaining questions.

We thank the reviewer for the valuable feedback.

Summary of main points:

• Q1: motivated the parallels between LLM and human associative learning by drawing on a line of prior works showing a functional equivalence between ICL and SGD.
• Q2: applied Benjamini-Yekutieli corrections across similarity groups and learning phases. Differentiation in mid-similarity range remains significant.
• Q4: showed that WordNet-restricted tokens exhibit monotonic representational change, consistent with high global interference. Strengths the point that NMPH depends on global similarity structure.
• Q6: fixed Fig 7a analysis; now shows monotonic differentiation across intermediate and late layers. Added new analysis using last-layer-optimized pairs (in progress).

Q1 [Parallels between human and LLM associative learning]: Thank you for raising this fundamental point. While it may appear unintuitive on the surface, we were motivated to investigate the possible parallels due to some recent successes in interpretability work on ICL. Several papers demonstrated that ICL performs updates similar to gradient descent [38, Reb2, Reb3, Reb4], suggesting that it may be closer to ‘synaptic weight updates’ than previously expected. Another paper drew a deep parallel between the mechanisms of induction heads in ICL to a well-validated model of human episodic memory (i.e. fast learning over a stream of episodes) [19]. We also do not think the field widely agrees that attention mechanisms are “completely different processes” than human learning for long-term information storage. Some work has drawn an analogy between attention mechanisms and the older concept of ‘fast weights’ in machine learning [Reb1][Reb2]. That is, one can view ICL as using attention layers to establish temporary weights that allow the LLM to flexibly learn patterns on a fast, short-term basis. While ICL does not directly modify the synaptic weights of the LLM, we do not think this invalidates the comparison with human learning. Rather, we believe the conceptual analogies of ‘synapse’ and ‘neuron’ break down when comparing very short-term learning between humans and LLMs.

Q2 [Statistical tests]: Thank you for raising this point. In our revised analysis, we apply Benjamini-Yekutieli correction to account for multiple comparisons across the 17 similarity groups and 3 learning phases. The findings remain significant, showing differentiation in the mid-similarity range (0.55-0.75). This is consistent with the original nonmonotonic pattern.

Q3 [Small sample size]: We extended the number of pairmates for 2 models (due to time constraints), sampling 100 pairs per similarity group, and found similar non-monotonic trends. We will add this to the revised paper.

Q4 [Semantically meaningful tokens]: Our study intentionally used token pairs selected for their pre-learning similarity, regardless of semantic meaning. This mirrors the use of synthetic stimuli in [39], which highlights the need to sample across the full similarity spectrum to test the NMPH effectively. Real-word tokens are unevenly distributed across this space, making precise control difficult. Our focus is on the structural dynamics of representational change during learning, not semantic meaning. That said, we agree that using meaningful tokens is important to assess whether NMPH applies in real-world contexts. To this end, we sampled new token pairs restricted to single-token words from WordNet, reducing the vocabulary to ~20% of the original token set in the two models tested so far. We first examined how pairwise similarity and vocabulary interference were distributed in this constrained space, anticipating that the limited vocabulary might bias the pairs toward narrower interference ranges. Indeed, real-word token similarity more closely aligned with the vocabulary interference distribution. As outlined in Sec. 3, this suggests that most of these pairs face high competition during prediction. Based on this, we hypothesized that representational change would shift from a U-shaped to a linearly decreasing pattern, as in the orange and blue curves of Fig 3b. Our results confirmed this: post-learning similarity decreased monotonically with pre-learning similarity, supporting the idea that highly similar pairs are influenced by vocabulary interference. We’ll include this new analysis in the revised paper and update our claims. Specifically, we’ll emphasize that NMPH is observed only under specific global interference conditions. Our findings suggest that to fully distinguish between competing hypotheses, global similarity structures, like vocabulary interference, must be considered as modulating factors.

Q5 [Examples of token pairs]: Here are randomly selected examples optimized for llama-7b. Similar examples for other models will be included in the appendix. Note: WordNet includes abbreviations and acronyms.

Synthetic token pairs:

Semantic token pairs:

Q6 [Intermediate layer analysis]: We appreciate you raising this concern. We will revise Fig. 7 to fix an analysis error and to add a new analysis. Since NeurIPS does not support figures or PDFs during the rebuttal, we summarize changes below:

1. Using token pairs optimized for similarity in earlier layers, similar to Fig 7a. Here we discovered an error in the code that indexed the wrong layers for averaging. When fixed, we saw:

• Intermediate and late layers show a monotonic pattern (e.g. similar to Fig 2b, green line), with significant differentiation for high similarity pairs (>0.7 pre-learning similarity.
• Intermediate layers show stronger differentiation (greater decreases in similarity) than late layers.
• Early layers (1 & 2) behave more erratically, lacking a clear trend.

2. New analysis using the same token pairs as Fig 2b (optimized for last layer similarity), replacing Fig 7b. This is still running, but we will update the rebuttal when our results come in. We note that in our setup, the learning process happens across repetitions, not across layers, so representational dynamics may change along the layer depth.

Q7 [Forgetting phase vs catastrophic interference]: While the "forgetting phase" in ICL might look like catastrophic interference, it's not as there are no weight updates. Instead, we attribute the accuracy drop to model-specific and reversible effects such as sliding window attention or vocabulary interference.

Q8 [Attention pattern]: While this is an important direction, we excluded it due to time constraints and our focus on representational dynamics. We see it as a valuable direction for future work.

[Reb1] J. Ba, G. E. Hinton, V. Mnih, J. Z. Leibo, and C. Ionescu, “Using Fast Weights to Attend to the Recent Past,” in Advances in Neural Information Processing Systems 29, 2016, pp. 4331–4339.

[Reb2] I. Schlag, K. Irie, and J. Schmidhuber, “Linear Transformers Are Secretly Fast Weight Programmers,” in Proceedings of the 38th International Conference on Machine Learning, PMLR, Jul. 2021, pp. 9355–9366.

[Reb3] Dherin, Benoit, et al. "Learning without training: The implicit dynamics of in-context learning." arXiv preprint arXiv:2507.16003 (2025).

[Reb4] Dai, Damai, et al. "Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers." arXiv preprint arXiv:2212.10559 (2022).

We hope we have addressed the reviewers' main concerns and we would happily go into more detail if there are any remaining questions.

Dear Reviewer ctgT,

Thank you for providing a thoughtful review. This is a reminder that the author-reviewer discussion period is about to close.

Did the authors reply address any of your initial concerns and are there any additional points the authors could clarify?