Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence

Thank you for your detailed review. We added experiments (notably Figures 21, 23, 24, 25, 26 and 27), in the following website: https://sites.google.com/view/in-context-meta-learning.

> W1

Experiment training with data of varying context lengths and next token prediction

We conducted additional experiments to address the concerns regarding generalizability.

• Context Length:
  In Figure 23, we tested varied context lengths (N=2 to 64). While longer contexts accelerate convergence to the FCC, the phase transitions remain observable up to N=8. In Figure 24, we show the circuit learned for N=16, which structurally align with the FCC reported in the main text.

• All-Label Prediction:
  In Figure 25, we experimented with predicting "all" labels in context. Phase transitions were still observed, and the final learned circuit again resembles the main FCC.

These results demonstrate that the key phenomena—phase transitions and FCC formation—are robust to variations in context length and prediction objective.

> W2

Experiments with more layers than two

Figure 26 shows experiments extending from 2-layer models to 3, 4, and 5-layer models. For the 2-layer and 3-layer models, we observe a phase transition in accuracy; however, when increasing to 4 or 5 layers, any phase transition in accuracy becomes less pronounced.
Additionally, Figure 27 visualizes the final circuits in the 3, 4, and 5-layer models. In all cases, the first two layers form a circuit resembling an FCC-like structure.

Our results show that even with 3, 4, and 5 layers, the key circuit still forms, though phase transitions in accuracy become less distinct. This demonstrates that our findings extend beyond a mere two-layer “toy” setup, indicating a robust mechanism likely relevant to larger-scale models.

> W3

Experiment with pre-trained foundation models

Please see W7 from Reviewer kgDv, where we show that a pretrained GPT2-XL exhibits chunk-example attention in its early layers and label-attention patterns in its middle-to-late layers on natural language data (see Figure 21). These findings align with our two-layer attention-only model and suggest that these circuits generalize to LLMs.

> W4

the paper extends the results of Reddy et al to the multi-task setup, which detracts from the overall novelty and significance.

We believe analyzing how Transformers operate in a more realistic few-shot in-context learning scenario is crucial, as few-shot performance is central to many practical LLM applications. While much of the existing circuit-focused research on in-context learning (e.g., induction heads) remains limited to copy tasks that are not seen in a practical LLM usage, our work explores circuit formation under a broader few-shot setup, offering a novel perspective. Furthermore, unlike the single-head approach in Reddy et al., we investigate multi-head architectures, shedding new light on how circuits differ in this setting. Finally, we also provide additional experiments with standard Transformers (see W3 from Reviewer kgDv) and pretrained-LLMs (see W3), underscoring the broader relevance and novelty of our analysis.

> W5

What are the 3 Tasks in T?

As explained in Section 3.1, each task $\tau$ defines a unique assignment of labels $\ell$ to items $x$. While the underlying classes and mean vectors remain the same, the labels are randomly reassigned from one task to another. Thus, the 3 tasks in $T$ are simply 3 different ways of pairing each class (and its mean vector) with a label, giving rise to three distinct label assignments under the same class/feature space.

> W6

Appendix C does not explain the pruning/masking of circuits used.

We believe the pruning/masking procedure is described in Appendix C, where we state: all components except for the circuit corresponding to a specific phase were pruned at initialization, thereby isolating the contribution of each circuit.

In Figure 12, the legends Phase1 Circuit, Phase2 Circuit, and Phase3 Circuit each refer to an experiment where only the circuit from the respective phase was retained from the initial weights, ensuring that only that circuit contributed to the predictions. Due to the limited space in the response, we will provide additional details on the exact pruning process for each circuit in the revised paper.

> W7

Does this rely on the model having memorized the two possible labels that can apply to xq

It is clearly the case that the model memorizes the two possible labels corresponding to xq. As explained in Section 4.1: Phase 1 (Non-Context Circuit; NCC): Both layers use bigram attention, ignoring the context and relying solely on the model’s weights. During Phase 1, the model does not utilize any contextual information, indicating that the model’s weights have memorized the two possible labels for xq.

Thank you for your detailed review. We revised the figures and added experiments (notably Figures 17 and 22) in the following website: https://sites.google.com/view/in-context-meta-learning.

> W1

The red block potentially make it even more hard to observe

We replaced the red squares with red arrows in Figure 17 and highlighted the updated caption in purple. Strong attention stands for entries that exhibit relatively high intensity in the row-wise softmax-normalized attention map. Note that the last query token often shows strong attention because it’s crucial for prediction.

Figure 2 is intended to illustrate the three distinct training phases (based on accuracy and loss) "qualitatively". This intuition from attention patterns motivates deeper analyses in the later sections. A more quantitative and rigorous analysis of attention maps appears in Figure 4 of the main paper.

> W2

K is not clear to me based on Sec. 3.1.

We will include the following table, which outlines the key hyperparameters and their default values, as well as a brief description of each parameter. We believe this format clarifies how the data is generated and represented in our experiments.

> W3

Missing a very related work

Thank you for noting [1]. Their results in pretrained models align with ours, especially regarding how label tokens gather context for deeper layers. Similar chunk-example attentions have also been noted in LLMs [2].

Unlike [1] and [2], we run controlled experiments to observe circuit formation during training.
We will clarify this connection and highlight how our dynamic analysis complements their findings.

[1] Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning.
[2] Revisiting In-context Learning Inference Circuit in Large Language Models.

> W4

the author constantly observe no transition happens on the accuracy curve.

In Figure 22, we show four training runs (seeds 0, 1, 2, and 3) of a 2-head attention model. Consistently, across all seeds, the accuracy curve does not exhibit clear, discrete phase transitions, supporting our observation that multi-head configurations tend to smooth out in accuracy.

> W5

I got confused here since the word "multi-head" suddenly jumped to me.

I went to appendix B but it does not tell me how many heads are used.

We will clarify that our default setting uses a single-head attention model. We then introduce the multi-head extension later to examine how additional heads affect phase transitions and circuit formation.

> W6

we do not know which phase real-world LLMs are in

We agree that it is difficult to conclusively determine which “phase” might dominate within a real-world LLM. However, our point is that the label-attention circuit seen in Phase 2 of our toy model—one that yields higher accuracy on random labels—may also be present (at least in part) in large-scale models. We do not claim LLMs remain in phase2, only that phse2 circuit likely contributes to their observed behavior on random-label prompts.

> W7

What is the gap between toy and practical settings?

In our toy setup, we observe abrupt phase transitions (e.g., a sudden drop in loss), whereas large-scale LLM training typically does not exhibit such discrete transitions in practice. When we introduce multiple heads, these abrupt transitions become smoothed out, making our toy model’s behavior more akin to real-world LLMs—thus “bridging the gap” between the simplified setting and practical scenarios.

> W8

Why the author sets L<K?

Please see W2 for our detailed revisions to Section 3.1. We set $L<K$ following prior studies [1], which introduce a notion of synonymous classes: multiple tokens can represent the same label. For example, the words “Happy” and “Glad” might both map to the same sentiment label. This design reflects more realistic linguistic variability.

[1] Data Distributional Properties Drive Emergent In-Context Learning in Transformers

> W9

The author could consider use \bm{\mu} for P3 col2 line130 since that's a vector.

We will modify the latter usage to indicate it is a vector, ensuring clarity in the notation.

We appreciate the reviewers’ insights and address each point below.

> W1

It seems to me that the RLA should be higher during SCC than during FCC.

Random Label Accuracy (RLA) can increase whenever label information is incorporated into the final token’s prediction. Even if the model uses a chunked example circuit (FCC), it can still leverage label information for that final prediction, which can maintain RLA. We do not claim that RLA must necessarily drop in the FCC phase or that RLA is definitively higher in SCC than in FCC.

> W2

The main weakness is incomplete discussion of prior work

We acknowledge that our study is related to multiple lines of research on in-context learning, multi-phase emergence. Below, we outline the connections and highlight where our approach differs.

---

1. ICL Literature Beyond Copy Tasks

• Linear Regression Approaches
  Prior studies on in-context learning often adopt linear regression [1,2], which provides a tractable theoretical framework. Although they set a common task across context examples, they typically rely on MSE loss—differing from real-world in-context use. Our setup more closely aligns with practical LLM applications.

• Markov Chain–Based Tasks
  Other works [3,4,5] focus on in-context learning with Markov chain tasks. While the meta-learning aspect is similar to our setting, these tasks differ from the few-shot pairs format that is standard in LLM applications.

• Modular Arithmetic
  Research on in-context modular arithmetic [6] studies out-of-distribution behavior and final attention maps. Although it extends beyond simple copy tasks, it does not track the circuit acquisition dynamics during training or directly link them to phenomena like random-label accuracy or multi-head smoothing—key aspects of our work.

---

2. Multi-Phase Emergence Literature

• Phase Transitions
  Prior work [4] shows transformers acquiring functionalities in discrete “phase transitions,” which aligns with our observations. However, the task there (Markov chains) is different from our few-shot setup, which is more akin to real-world LLM usage. Also, while [4] studies a progression from uniform prediction → unigram → bigram, we analyze more complex Bigram → Semi-Context → Full-Context circuits.

• Developmental Interpretability
  Studies like [7,8] also note multi-phase transitions in in-context learning. However, they mainly focus on the loss landscape’s geometry (e.g., local learning coefficient, or LLC) rather than the mechanistic circuits we analyze. Bridging LLC-based approaches with an internal-circuit perspective could be an exciting future direction.

We will include those discussions and citations in the revision.

[1] Transformers Learn In-Context by Gradient Descent.
[2] Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression.
[3] Selective induction Heads: How Transformers Select Causal Structures in Context.
[4] The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains.
[5] Competition Dynamics Shape Algorithmic Phases of In-Context Learning.
[6] Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks.
[7] The Developmental Landscape of In-Context Learning.
[8] Loss Landscape Degeneracy Drives Stagewise Development in Transformers.

> W3

I invite the authors to consider a different title for this section to avoid possible confusion.

We will rename that section to “Multi-Head Attention Facilitates the Emergence of Distinct Circuits” to avoid conflating our focus on circuit formation dynamics with mechanistic-interpretability efforts on circuit discovery.

> W4

While I generally agree with the framing that meta-learning is an important element of ICL, I didn't find this example compelling.

We agree that induction heads can perform pattern matching at a more abstract level, as described in the original induction heads paper (Argument 4 in Olsson et al.). However, consider a scenario where the token “Tokyo” does not appear in the context: if the model only performs a (possibly abstract) copy operation, it cannot generate “Tokyo” from the context itself. In other words, merely copying from the examples—whether literally or via a higher-level “abstract match”—is insufficient for a task like “Find the capital of Japan” unless “Tokyo” is explicitly present in the context.

Thus, our central motivation remains unchanged: while induction heads can extract answers that are already present in the context, they do not fully explain the ability of LLMs to infer a shared task (like a country-to-capital mapping) and then apply it to a query whose answer is not directly provided in the context. Numerous studies (including research on task vectors) suggest that LLMs can learn and apply such tasks in-context beyond the scope of mere copy-based or induction-head operations.

Thank you for your detailed review. We revised the figures and added experiments (notably Figures 17, 18, 19, 20 and 21) in the following website: https://sites.google.com/view/in-context-meta-learning.

> W1

attention maps in Figure 2, it is not simple to identify the circuits.

Please see Figure 17, and refer to W1 from Reviewer bYju for more details.

> W2

Is the accuracy calculated on test data?

We focus on training dynamics and internal circuits in a practical task rather than OOD behavior as in [1]. Hence, we do not strictly separate train and test sets, although section 4.3 examines OOD behavior via random labels. For completeness, Figure 18 presents test accuracy, confirming that phase transitions also manifest.

[1] “The mechanistic basis of data dependence and abrupt learning in an in-context classification task”

> W3

The considered architecture is peculiar.

Using a two-layer standard Transformer, interleaving attention and MLP blocks, we still see phase transitions in accuracy and observe similar circuits (SCC,FCC) in Figure 19. However, the transitions are less distinct. We adopted a two-layer, attention-only Transformer to focus on self-attention, whose architecture is widely used in prior works analysing in-context learning [1,2,3]. The minimal models clearly simulates in-context behaviors such as induction heads [4], and can be useful for clearer phase-transition and circuit-level analysis.

[2] “What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation”
[3] “Differential learning kinetics govern the transition from memorization to generalization during in-context learning”
[4] “In-context Learning and Induction Heads”

> W4

some metrics present some overlap which are not discussed in depth.

We address why Chunk Example and Bigram remain high outside their respective phases:

---

1. Chunk Example
   Using the chunk-example metric (Table 2), the initial value is $\frac{1}{4}(\frac{1}{2}+\frac{1}{4}+\frac{1}{6}+\frac{1}{8})\approx 0.26,$which we treat as the baseline.

• In Phase 1, both Layer 1 and 2 stay near this baseline.
• In Phase 2, the metric increases in Layer 2 but does not affect the final output because the loss is only computed on the last token.
• In Phase 3, the metric in Layer 1 exceeds the baseline and influences the final output, showing that the chunk-example circuit is forming and impacting the last token’s prediction.

---

2. Bigram

• In Phase 2, due to row-wise softmax normalization, strong attention to label tokens in Layer 1 diminishes Bigram attention.
• In Phase 3, in contrast, Layer 1’s chunk example pattern does not compete with Bigram, allowing the Bigram metric to recover relative to its Phase 2 level.

We will add this explanation in the revised version.

> W5

what happens when T grows larger

We tested $T=3,6,9,12,15,18$ (see Figure 20) and found that phase transitions still occur for larger $T$. Increasing $T$ reduces Phase 1 accuracy (around $1/T$) and delays the transition to the next phase.

> W6

section 3.1 would benefit from a clearer explanation

Please see W2 from Reviewer bYju.

> W7

more standard architectures and in larger models.

For standard Transformer results, please see W3.
We tested whether our identified circuits appear in LLMs by evaluating a pretrained GPT2-XL (48 layers) on SST2 (872 samples). We used a 2-shot prompt: two labeled examples (Review: {text}\nSentiment: {label}) and a third, unlabeled query. Since the model is already trained, we observe each layer’s behavior on these prompts.

We measure 3 metrics by averaging attention $p(i,j)$ across all heads in each layer, where $p(i,j)$ denotes the attention from the $j$-th token to the $i$-th token:

1. Bigram $p(\text{query}, \text{query})$
   Here, query refers to the final token, and this metric measures a query token’s attention to itself.
2. Label Attention $\frac{1}{K}\sum_{k=1}^{K} p(\text{query}, \text{label}_k)$
   Where $K$ is the number of shots, and $\text{label}_k$ is the label token in the $k$-th example. This captures how strongly the query token attends to label tokens.
3. Chunk Example $\frac{1}{K}\sum_{k=1}^{K} \frac{\sum_i p(\mathrm{label}_k, \mathrm{text}_k^i)}{n_k}$
   Here, $n_k$ is the number of text (non-label) tokens in the $k$-th example, and $\mathrm{text}_k^i$ represents the $i$-th such token. This metric reflects how strongly each label token attends to the text tokens in its corresponding example.

Figure 21 shows that the Chunk Example metric is higher in early layers, while Label Attention dominates later layers—mirroring our two-layer model’s progression from chunk example to label focus. This pattern aligns with our earlier findings in small Transformers, suggesting these circuits generalize to LLMs.