Scaling and context steer LLMs along the same computational path as the human brain

We thank Reviewer eQDJ for their thorough review. We propose to amend our manuscript to address these issues and add five additional sets of results.

1. Pearson correlation

These Pearson R scores are indeed low, but remain in the range typically observed with electrophysiology. For example, Caucheteux and King (2022) report, from MEG results, brain-scores around R=0.07, peaking at R=0.13 for the best channels. Goldstein et al (2023) report, from intracranial data R=0.2 on the electrodes specifically selected to show language responses. These values are low because (1) the signal-to-noise ratio in electrophysiology is low and (2) there is no denoising procedure (e.g. averaging across repetitions, normalizing by estimated noise ceiling etc). We will add a figure with confidence intervals to illustrate that in spite of being small, these effect sizes are highly reliable.

• Caucheteux & King (2022), Communications Biology. Brains and algorithms partially converge in natural language processing.
• Goldstein et al. (2022), BioRxiv. Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain

2. Shared objective vs representations

This is an interesting remark. To address it, we now amended the discussion as follows:

As previously reported (Jain et al, (2020), Caucheteux and King (2022), Toneva and Wehbe (2019), Antonello et al. (2023)), the best brain scores are achieved in the intermediary layers. This result suggests that the intermediary representations (layer=60% of architecture) – as opposed to the input and the prediction – are best aligned between brains and LLMs, suggesting that these two systems don’t just share the same objective, but the same computational path too.

• Jain et al. (2020), NeurIPS. "Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech."
• Caucheteux & King (2022), Communications Biology. "Brains and algorithms partially converge in natural language processing."
• Toneva & Wehbe (2019), NeurIPS. "Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)."
• Antonello et al 2023 NeurIPS. "Scaling laws for language encoding models in fMRI."

3. New experiments: Additional models and analyses

To address reviewers’ comments, we now run five additional analyses:

1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to causal models previously showcased in the manuscript, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.

2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.

3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.

4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.

5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). The results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.

Temporal Score

Brain score

We would like to thank again Reviewer eQDJ for their insightful comments, and hope this further strengthens our contribution.

Dear reviewer, in light of the added experiments and revised sections, is there any additional element you believe we could add or discuss to address your review?

We thank Reviewer tqrA for their thorough review. We propose to amend several sections of the manuscript to address these issues and add five additional sets of results.

1. Contribution

We agree that several studies had shown a temporal alignment between brain and LLM before – although it is here demonstrated with 20x more data – 10h per participant, compared to 30 min per subject – and with minimal preprocessing – e.g. no localizer or analyses restricted to clusters or regions of interest – compared to both Li et al. (2023) and Goldstein et al. (2022).

• Li et al. (2023), Nature Neuroscience. Dissecting neural computations in the human auditory pathway using deep neural networks for speech
• Goldstein et al. (2022), BioRxiv. Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain

Rather, we start from this independent replication to :

(1) Systematically compare 23 diversified language models to brain activity.

(2) Use this benchmark to identify the factors that cause this temporal alignment (architecture, model size, training quantity, directionality, context length).

(3) Successfully test the correlation between higher brain scores and higher temporal alignment, identifying conditions where brain score plateaus while temporal score continues to rise (cf "6. New Experiments" for further details).

While we believe this contribution is accurately stated in the introduction and discussion, we do agree that the abstract was insufficiently clear on our exact contribution. We thus propose to amend it as follows:

• “However, whether this representational alignment arises from a similar sequence of computations remains elusive. “ -> “However, whether and why this representational alignment arises...”
• “Our analyses reveal that LLMs and the brain generate representations in a similar order” -> “Our analyses confirm ...”
• “Overall, the alignment between LLMs and the brain provides novel elements supporting a partial convergence between biological and artificial neural networks.” -> “Overall, this study clarifies the factors underlying the partial convergence between ...”

2. Credit to scaling laws studies

We agree that we insufficiently credited previous work on scaling laws. We now amended the discussion as follows:

Third, our experiments show that this alignment directly depends on (i) context size and (ii) model size. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)). In particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.

• Antonello et al 2023 NeurIPS. Scaling laws for language encoding models in fMRI.
• Bonnasse-Gahot & Pallier (2024), NeurIPS. fMRI predictors based on language models of increasing complexity recover brain left lateralization
• Tikochinski et al. (2025), Nat. Comm. Incremental accumulation of linguistic context in artificial and biological neural networks
• d’Ascoli et al. (2024), arXiv. Decoding individual words from non-invasive brain recordings across 723 participants
• Banville et al. (2025), arXiv. Scaling laws for decoding images from brain activity
• Caucheteux et al. (2023), Nat. Hum. Behav. Evidence of a predictive coding hierarchy in the human brain listening to speech

3. Expected results

We agree that models with many more layers are likely to have an ‘elongated processing pipeline’. However, this does not affect our analysis, because we sample the same number of layers (n=9, linearly space between 10% and 90% of the architecture) for each model. In addition, to clarify that our results may not be fully expected by the community, we now highlight papers that emphasize the discrepancies between LLMs and the Brain:

Yet, their architecture, sensory modality, learning objective, and training regime are remarkably disjoint Romeo et al. (2018), Dupoux (2018), Evanson et al. (2025), Ghanizadeh & Dousti (2025). Furthermore, specific features like syntactic structures and semantic roles may be represented fundamentally differently (Fodor et al. (2025), Kryvosheieva & Levy (2024)). That partially converging computational paths arise despite these differences is thus all the more surprising, and highlights the necessity to clarify the computational principles that lead language processing to be partially shared between biological and artificial systems.

• Romeo et al. (2018), JNeuro. Language exposure relates to structural neural connectivity in childhood
• Dupoux (2018), Cognition. Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner
• Evanson et al. (2025), Manuscript. Emergence of language in the developing brain.
• Ghanizadeh & Dousti (2025), arXiv. Towards data-efficient language models: A child-inspired approach to language learning
• Fodor et al. (2025), CL, Compositionality and Sentence Meaning: Comparing Semantic Parsing and Transformers on a Challenging Sentence Similarity Dataset
• Kryvosheieva & Levy (2024), arXiv. Controlled evaluation of syntactic knowledge in multilingual language models

4. Brain-score averaging and per sensor analysis

The brain-score is indeed the result of Pearson scores independently computed for each sensor, then averaged across the brain. We will state this averaging in the Methods section.

Adding the per-sensor analyses is indeed a good idea. We now have these encoding results available and will add them to the manuscript. Note that the low spatial resolution of MEG remains challenging to explore this issue.

5. New experiments beyond LLMs: Wav2Vec2

Note that we already use two types of architectures: State Space models (can be thought of as RNNs with a linear dynamics of their hidden state) and Transformers.

We now include results for speech model Wav2vec2.0, bidirectional, alongside bidirectional LLMs BERT and RoBERTa. All three show brain-scores comparable to larger, more recent and causal LLMs, but much lower temporal scores. These results are shown in the first table of the next section, alongside additional analyses.

6 New experiments: Additional models and analyses

To address reviewers’ comments, we now run five additional analyses:

1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to the causal models previously presented, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.

2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.

3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.

4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.

5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (SSM). The results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.

Temporal Score

Brain score

Together these MEG results are consistent with the overall trends observed with fMRI (Antonello, et al (2024).

We thank Reviewer tqrA again for these constructive criticisms which help strengthen this study, and better articulate how it fits the current state-of-the-art.

Dear reviewer, in light of the added experiments and revised sections, is there any element you believe we could add or discuss to address your review?

We will indeed add these changes to the paper. Thank you for your comment and thorough review.

We thank Reviewer WHgU for their thorough review. We propose to amend our manuscript to address these issues and add five additional sets of results.

1. New experiment with large language models

We now managed to implement our analysis also on model Llama 3 Instruct (70B). The obtained results don’t outperform results from smaller models, which suggests a limit to the scaling trends observed earlier. We now indicate in the discussion section:

Third, our experiments show that this alignment directly depends on (i) context size and (ii) model size – although with a saturation beyond 70B parameters models. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)). In particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.

• Antonello et al 2023 NeurIPS. Scaling laws for language encoding models in fMRI.
• Bonnasse-Gahot & Pallier (2024), NeurIPS. fMRI predictors based on language models of increasing complexity recover brain left lateralization
• Tikochinski et al. (2025), Nature Communications. Incremental accumulation of linguistic context in artificial and biological neural networks
• d’Ascoli et al. (2024), arXiv. Decoding individual words from non-invasive brain recordings across 723 participants
• Banville et al. (2025), arXiv. Scaling laws for decoding images from brain activity
• Caucheteux et al. (2023), Nature Human Behaviour. Evidence of a predictive coding hierarchy in the human brain listening to speech

2. New experiment on Mamba with context size

We now extend our context size experiments to Mamba up to n=1,000 words, where temporal score keeps increasing until 0.93. As comparison, we conduct the same analysis for Llama-3.2, where temporal scores also keep increasing across increasingly long contexts. These elements suggest, with MEG, the very long context sizes do still improve the modeling of brain activity for at least some of the most recent models, State-Space and LLMs. Results are all showcased and contextualized alongside other analyses in the table of 4. New experiments, part 5).

3. New experiments on old models and RNNs

We now extended our pipeline to older language models including GPT-2 small, BERT and Roberta. These older language models present brain-scores comparable to larger, more recent LLMs, but much lower temporal scores. These results are shown and contextualized in the first table of the next section 4. New experiments, with additional analyses.

We also analyze State-Space models (Mamba and RecurrentGemma), which can be viewed as types of RNN with a linear evolution of the hidden state at each layer. Additionally, we propose to add the analysis of a recent RNN, xLSTM (Beck et al 2024), in the final version of the manuscript.

• Beck et al. (2024), NeurIPS. "xLSTM: Extended Long Short-Term Memory."

Overall, the updated experiments do not change but extend further our original conclusion.

4. New experiments: Additional models and analyses

To address reviewers’ comments, we now run five additional analyses:

1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to causal models previously showcased in the manuscript, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.

2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.

3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.

4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.

5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). The results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.

Temporal Score

Brain score

We would like to thank again Reviewer WHgU for their insightful comments, and hope this further strengthens our contribution.

Dear reviewer, in light of the added experiments and revised sections, is there any additional element you believe we could add to address your review, or something you would wish to discuss?

We thank Reviewer mo9D for their thorough review. We propose to amend several sections of the manuscript to address these issues and add five additional sets of results.

1. Discussion working memory

This is indeed a very good point. We will add the following paragraph:

“Interestingly, the comparison between State Space Models (SSMs) and LLMs, offer a new perspective to investigate context-size. Indeed, transformers compute contextual representations, thanks to a non linear combination of the past context, and hence require very large memory buffers that are implausible in the brain. By contrast, SSMs can be thought of as recurrent neural networks (RNNs) with hidden states that linearly evolve over time. At each time point, they thus represent the full context with their hidden state – an approach presumably similar to the brain. Our results show that SSMs do present very large context size improvements, and thus show that it is, in fact possible, to build and maintain a long context in a single hidden state. However, further research remains necessary to evaluate the degradation of such memory in the absence of meaningful context, such as during the memorization of a random digit sequence like a phone number – a task recognized as highly constrained in human subjects (Miller et al. (1956)).”

• Miller (1956), Psychological Review. "The magical number seven, plus or minus two: Some limits on our capacity for processing information."

2. Discussion model scaling

We agree that we insufficiently credited previous work on scaling laws in neuro-ai. We now amended the discussion as follows:

Third, our experiments show that this alignment directly depends on (i) context size and (ii) model size. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)). In particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.

• Antonello et al. (2023): Antonello et al 2023 NeurIPS. "Scaling laws for language encoding models in fMRI."
• Bonnasse-Gahot & Pallier (2024): Bonnasse-Gahot & Pallier (2024), NeurIPS. "fMRI predictors based on language models of increasing complexity recover brain left lateralization."
• Tikochinski et al. (2025): Tikochinski et al. (2025), Nature Communications. "Incremental accumulation of linguistic context in artificial and biological neural networks."
• d’Ascoli et al. (2024): d’Ascoli et al. (2024), arXiv. "Decoding individual words from non-invasive brain recordings across 723 participants.",
• Banville et al. (2025): Banville et al. (2025), arXiv. "Scaling laws for decoding images from brain activity."
• Caucheteux et al. (2023): Caucheteux et al. (2023), Nature Human Behaviour. "Evidence of a predictive coding hierarchy in the human brain listening to speech."

3. Redesigned figure 6

We agree with this note and propose to separate the figure 6a in three plots. The line linking the markers is not crucial to the figure but its objective is indeed to highlight trends where bigger models and longer contexts induce higher brain-score and temporal-score. Separating the figure in three plots shall make these trends clearer.

4. Baselines as untrained or less competent model such as GPT-2

The untrained versions of the models are shown as dashed grey lines in Figure 1 of the manuscript.Their brain scores did not differ significantly from zero over time, resulting in a null temporal score.

Analyzing a less competent models such as GPT-2-small is indeed a very good point. We conducted this analysis and found a brain-score of 0.054, comparable to more recent and bigger models, but a temporal score of 0.20, which is substantially lower. We further analyze and contextualize this result in the following section, alongside additional analyses.

5. New experiments: Additional models and analyses

To address reviewers’ comments, we now run five additional analyses:

1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to the causal models previously presented, more closely reflecting the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.

2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.

3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.

4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.

5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). The results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.

Temporal Score

Brain score

We would like to thank again Reviewer Mo9D for their insightful comments, and hope this further strengthens our contribution.

Thank you for your response ! As we implemented the experiments you proposed, please let us know if there is anything else you would like to discuss, or an additional element you think we could add to increase your rating.