Toward a Mechanistic Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model

To summarize, the goal of our paper was to use graph navigation as an abstraction of tasks where stepwise inference is known to play a critical role and correspondingly develop and validate precise hypotheses that can help understand stepwise inference. This is similar to several recent works studying synthetic problems for developing a better understanding of LLMs [4,5, 6, 7, 8,9,10]. As we note above, we believe our abstraction is faithful and hence we expect our results will transfer to realistic scenarios. Such an investigation is best left to future work, however. We also emphasize that to further clarify the comments above, we have rewritten parts of the introduction where we have elaborated our exposition on the relation between stepwise inference and graph navigation, rewritten our abstract and also edited the formatting and arrangement of figures and results in paper (including promoting the training dynamics figure the main text since it shows that global planning capability arises after models learn to take correct steps). We hope this has clarified the contributions and limitations of our work, but we are happy to discuss further if you have any remaining concerns.

[1] Mommenejad et al. Evaluating Cognitive Maps in Large Language Models with CogEval: No Emergent Planning (NeurIPS 2023)

[2] Saparov et al. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (ICLR 2023)

[3] Dziri et al. Faith and Fate: Limits of Transformers on Compositionality (NeurIPS 2023)

[4] Hou et al. Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models. (arXiv, 2023)

[5] Zeyuan Allen-Zhu and Yuanzhi Li, (2023) Physics of Language Models: Part 3.1, Knowledge Storage (2023)

[6] Zeyuan Allen-Zhu and Yuanzhi Li, Physics of Language Models: Part 3.2, Knowledge Manipulation (2023)

[7] Maya Okawa et al. Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task (NeurIPS 2023)

[8] Zhou et al. What Algorithms can Transformers Learn? A Study in Length Generalization. (arXiv, 2023; https://arxiv.org/abs/2310.16028)

[9] Feng et al, Towards demystifying the mystery behind chain of thought: A theoretical perspective (arXiv, 2023)

[10] Li et al. Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning (NeurIPS 2023)

We thank you for your review and glad to hear that you find our idea of modeling step-wise inference as graph navigation interesting. We address your specific concerns below.

Comment: Although the authors claim to reveal the connection between graph navigation and the step-wise inference mechanism in transformer, the experiments, to my understanding, are solely about training transformers on the synthesized graph dataset. It remains unclear to me how this set-up can be translated into the study of step-wise inference mechanism, despite the conceptual similarity between DAG and the step-wise inference mechanism. It is unclear whether this finding on the synthesized graph dataset can be safely transferred to real-world large-scale dataset.

Response: Thank you for this question, which touches on very important aspects of the motivation, objectives, and limitations of our work. Your question has prompted us to add the new paragraph at the end of the Introduction on "The motivation, contributions, and limitations of our model-experimental systems approach." (Please also see our global response for related discussion as well.) To provide further evidence for the correctness of the analogy between our synthetic graph navigation task and stepwise inference tasks that real LLMs are evaluated on, we have also created a new Appendix Figure 8, where we showcase several recent works [1,2,3,4] that have studied the impact of stepwise inference on tasks that can easily be modeled in our graph navigation framework. We elaborate key arguments below.

First, our contributions can be divided into three main parts of (i) formulation of synthetic graph navigation model motivated by the analogy between DAG and the step-wise inference; (ii) empirical validation of the model by precise reproduction of phenomena observed in LLMs; and (iii) discovery of new mechanistic hypotheses such as the impact of the underlying graph structure on stepwise inference, path stitching mechanisms, diversity-accuracy tradeoff, shorter-path bias in inference, and more.

In response to your specific concerns, beyond the "conceptual similarity between DAG and the stepwise inference mechanism", we have experimentally validated our model by reproducing an array of phenomena observed in LLMs. Furthermore, while all experimental results are obtained by "training transformers on the synthesized graph dataset", as you correctly point out, we believe that our novel mechanistic hypotheses are our important contribution. Finally, we agree with the necessary trade-offs and limitations of our study that our proposed hypotheses are not conclusive about the mechanisms underlying LLMs. We have clarified this in our new paragraph as, "To draw an analogy to the study of biological neural networks, where neural mechanisms identified in small-scale model organisms such as fruit flies or mice may not be directly applicable to medical applications involving the human brain, our observations should not be taken as definitive conclusions that are readily applicable to modern large-scale generative models.” Nevertheless, we are hopeful that our novel hypotheses can translate to larger-scale settings, based on the fact that our model system, despite its simplicity, successfully reproduced an array of known phenomena in LLMs. Testing them would require training and evaluating models on larger-scale setup, but would be an exciting future direction.

Comment: The definition of hierarchical graph and random graph is confusing to me. It seems possible to convert the random graph in fig. 10 into a hierarchical graph by simply re-grouping the nodes.

Response: In a hierarchical graph, nodes are pre-partitioned into layers. The only connections allowed are between successive layers and these are chosen randomly. Whereas in a random graph, all connections are chosen at random. It is not possible to simply regroup nodes in a random graph and convert it to a hierarchical graph. To provide evidence for this, we now refer the reviewer to Appendix Figure 9 from our paper, where we highlight and discuss the differences between a hierarchical and random graph. In that figure, the bottom panels show the distribution of path lengths in the two families of graphs: hierarchical graphs have an exponential distribution owing to their structure and random graphs have a gaussian distribution. For example, given any pair of nodes, in a hierarchical graph every path between them is of the same length (determined by the difference of their layer numbers), whereas in a random graph there is a diversity of path lengths (for instance, there might be an edge between the nodes as well as a path of length more than 10, as is the case with our synthetic random graphs). We wish to emphasize that our specific approach to using a hierarchical graph is designed to enable more precise control over the data distribution.

To further ensure this point is clear, we have added pseudocode for the algorithms for generating both the random and hierarchical graphs—Please refer to Algorithms 1 and 2 in the Appendix A.3. We hope that this,along with the path characterization provided in Figure 9, addresses concerns about possible similarities between the two setups.

Comment: The highlighted finding, i.e., the step-wise inference gap is influenced by (i) the underlying DAG structure, and (ii) the length of the training samples is only studied in a very shallow level. The DAG structure in the context seems to be not well-defined and well-categorized. The discussion about the length of the training samples mainly focuses on how it affects the length of the model output.

Response: Our highlighted findings in a single graph scenario include: (i) the inference gap consistently persists across diverse data distributions in the training data (Figure 3); (ii) a diversity versus accuracy trade-off exists in finite temperature stepwise inference for transformers (Figure 4); and (iii) demonstrating that there is a bias towards shorter paths and the observation that over the course of training the model first learns to take correct steps and then acquires the capability of global planning (Figure 5). We recognize that our approach serves as a first step towards formulating a set of hypotheses for further validation in large-scale, realistic settings, and there is considerable scope for extending our research to more realistic settings such as different data distributions, noisy or incomplete information, and different graph structures. As part of this endeavor, We have now repeated our analysis on a setup where a ground-truth graph is “noisily observed”. Specifically, every time a path is sampled between two nodes, we intentionally corrupt a random fraction of the tokens on the path by 5-20%. The model is now trained on these corrupted paths—this randomization further ensures that overfitting is infeasible, since a given path is never seen twice (due to corruptions). In this setup, we again see that our claim on the performance gap induced by stepwise inference continues to persist. We have also defined our training protocol and description of the graphs used in training in more detail (in Appendix).

Summary: We appreciate your constructive feedback, which helped us to clarify the motivation, objectives, and limitations of our synthetic data approach. We hope that the contributions of our work are now clearer and that the reviewer will consider raising their score to support the acceptance of our work.

Thank the authors for their efforts to continue this discussion.

I would like to again point out that first, my suggestion about using synthetic language datasets originates from the following two concerns: i) the lack of significant theoretical contribution other than the direct application of the definition of DAGs (if we were to verify the hypotheses on synthetic graph data as in [1, 2]), and ii) in addition to i), the non-negligible gap between the inputs of the current framework and those actually used for LLMs, i.e., insufficient empirical evidence for safely transferring the current findings to real-world tasks. These two concerns together undermine my confidence of the current submission being acceptable to a top conference like ICLR.

I thank the authors for further pointing me to [3], [4] and [5]. While [3] is surely an interesting piece of work using only synthetic data with few or no theoretical deduction, the hypotheses are examined with extensive, comprehensive and well-designed experiments which I highly appreciate. These experiments are scaled up and deliberated designed to closely approximate the richness of real language (e.g., see Sec. 2.2 Why Such CFGs in [3]) unlike those in the current submission. [4] and [5], however, are somehow unrelated to the current topic of our discussion, in my opinion. Although [4] did use synthetic data to probe the compositional generalization ability of conditional diffusion models as the core motivating examples and results, there is a section, i.e., `Sec. 4.2 Additional Experimental Results with Real Data' to showcase that the observations are truly consistent (at least partially) with real data. I am admittedly not very familiar with the area of causal reasoning, which is the case in [5] and the case is seemingly not very related to the current problem.

Again, I appreciate the efforts the authors made to try to address my concerns and think the idea of modeling step-wise inference as graph navigation is interesting. It is not likely, however, that I will change my opinion with more listed related/unrelated works if the main concerns, which I believe are clearly stated, are not addressed. I respect the endeavors and opinions from other reviewers, and honor the final decision made by the committee no matter what that will be.

We thank the reviewer for continued engagement.

I suggested using at least synthetic language datasets as used in the referred works,

Firstly, we'd like to point out that [1] and [2] did not use any language datasets in their proofs or experiments. [1] uses a similar setup to ours where tokens correspond to nodes on a graph and [2] uses random 4-64 dimensional euclidean vectors.

Secondly, while [1] and [2] have a fraction of theory results, a majority of synthetic data work does not. As examples from diverse areas, consider: [3] which studies how transformers navigate small tree-structured graphs from CFGs, [4] which studies a synthetic image dataset in diffusion models and empirically shows the delayed emergence of compositionality and [5] which studies RL agents interacting with simple causal DAGs with 5 nodes, with purely empirical results and no theorems with proofs. These are most similar in style to our work, which studies "empirical models" , first validates their approach by reproducing phenomena and then produces hypotheses to test at scale.

Our explicit list of hypotheses and contributions is given at the end of the introduction, just before discussion of our limitations and each hypothesis is stated in the corresponding results section. To summarize everything together:

• We find that the Step-by-Step Inference gap depends on two key factors, firstly the structure of the underlying DAG: the Step-by-Step Inference gap is larger for graphs that are hierarchical as opposed to random and secondly, the length of the training samples: the Step-by-Step Inference gap is larger if the model has been trained on a set of shorter paths that have to be ``stitched" together to build the path during evaluation.

• We find that models have a shorter path bias in the case of high path diversity.

• We systematically and quantitatively characterize the diversity-accuracy tradeoff, showing curves with loss of accuracy and increase of diversity as a function of sampling temperature.

• Over the course of training, we find that global planning capabilities arise after the model has learnt to take correct steps

• We hypothesize that for chain-of-thought and related methods at scale: the model will fail to generalize to reasoning chains longer than those present in its training data.

• When presented with 2 conflicting chains in-context, we find that the model has a strong bias toward choosing the first chain over the second.

[3] Allen-Zhu et al. Physics of Language Models: Part 1, Context-Free Grammar https://arxiv.org/abs/2305.13673

[4] Maya Okawa et al. Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task. NeurIPS 2023 (https://arxiv.org/pdf/2310.09336.pdf)

[5] Ishita Dasgupta et al. Causal Reasoning from Meta-reinforcement Learning ICLR 2019 (https://arxiv.org/abs/1901.08162)

Comment: Emergent abilities of large language models were thought to be a consequence of unpredictable scaling, therefore, I doubt if a simpilified DAG substitute can replicate this behaviour.

Response: We respectfully disagree with the claim that emergent abilities only occur in large scale models. Abilities like in-context learning (ICL) and stepwise inference protocols like chain-of-thought (CoT) have been shown and extensively studied in small toy models by several recent works by designing well defined synthetic tasks [1,2,3,4,5,6,7,8,9,10]. The goal of such studies (and ours well) is to study interesting phenomenology of LLMs and design abstract setups that enable development of precise mechanistic hypotheses for how the phenomenon works. For example [4,5] use a synthetically designed setup to develop hypotheses for how ``knowledge” about an entity is stored in a pretrained model, showing that such knowledge can be manipulated via relatively simple linear transformations. This development of mechanistic hypotheses is precisely what we have done with our experiments.

We also note that to ensure the point about relation to LLMs is addressed in the paper, we have substantially revised our manuscript to further motivate the explicit connection between graph navigation and stepwise inference in more realistic settings. Specifically, we have rewritten parts of the introduction (we have elaborated our exposition on the relation between stepwise inference and graph navigation, added a discussion of the limitations of a synthetic data approach), rewritten our abstract, contextualized our results more thoroughly with existing literature (added Appendix figure 8 which shows examples of graph navigation in a LLM), and also edited the formatting and arrangement of figures and results in paper (including promoting the training dynamics figure to the main text since it shows that global planning capability arises after models learn to take correct steps). As an example, our edits now highlight several recent studies that either use or model stepwise inference with LLMs to study tasks that are specific instantiations of our proposed graph navigation problem (e.g., see [11,12, 13, 14]). We have included a figure in Appendix (Fig. 8) that describes example prompts and model outputs from each of these papers to exemplify the relation of our proposed synthetic benchmark to realistic setups and tasks considered by prior work in showing the value of stepwise inference. Given that these studies exist in the (very recent) literature, we argue the connection between stepwise inference, planning, and graph navigation is explicit and our updated draft ensures this connection is emphasized upon.

With our detailed responses and changes to the paper, we hope that we have addressed the reviewer's concerns and the reviewer increases their score.

[1] Bai et al. Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection (NeurIPS 2023)

[2] Akyurek et al. What learning algorithm is in-context learning? Investigations with linear models (2022) https://arxiv.org/abs/2211.15661

[3] Ahn et al. Transformers learn to implement preconditioned gradient descent for in-context learning (NeurIPS 2023)

[4] Zeyuan Allen-Zhu and Yuanzhi Li, (2023) Physics of Language Models: Part 3.1, Knowledge Storage (2023)

[5] Zeyuan Allen-Zhu and Yuanzhi Li, Physics of Language Models: Part 3.2, Knowledge Manipulation (2023)

[6] Maya Okawa et al. Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task (NeurIPS 2023)

[7] Lindner et al (2023) Tracr: Compiled Transformers as a Laboratory for Interpretability https://arxiv.org/abs/2301.05062

[8] Feng et al, Towards demystifying the mystery behind chain of thought: A theoretical perspective (2023)

[9] Li et al. Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning (NeurIPS 2023)

[10] Prystawski et al. Why think step by step? Reasoning emerges from the locality of experience (NeurIPS 2023)

[11] Mommenejad et al. Evaluating Cognitive Maps in Large Language Models with CogEval: No Emergent Planning (NeurIPS 2023)

[12] Saparov et al. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (ICLR 2023)

[13] Dziri et al. Faith and Fate: Limits of Transformers on Compositionality (NeurIPS 2023)

[14] Hou et al (2023) Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models https://arxiv.org/pdf/2310.14491.pdf

We thank you for your review and enthusiasm about our work! We are happy to hear that you think our framework is comprehensive, our experiments were well-designed and that our paper was well-written. We respond to specific comments below.

Comment: While several phenomena originally observed in LLMs were replicated in this DAG couterpart. It will be more convincing if some extrapplated property in DAG system can be found in LLMs which was not discovered before.

Response: We appreciate the reviewer’s concern that we might have only replicated phenomena at scale. We would like to take the opportunity to clarify that our study does more than mere replication of phenomena already observed in LLMs.

Firstly, we ground and validate our synthetic setup by replicating phenomena observed at large scale, namely the step-wise inference gap.

Next, we go beyond by: (i) identifying new phenomena leveraging the fact that we have an interpretable ground truth in the data-generating process. Specifically, we have found and characterized the following phenomena:

• The dependence of the stepwise inference gap on the structure of the underlying graph

• A short-path bias when there exist several paths between a pair of nodes

• The diversity-accuracy tradeoff

• The emergence of global planning capability after learning to take correct steps over training

• The dependence on the length of the context during training to generalization during evaluation

• The model’s bias towards specific examples when there exist conflicting examples in context

(ii) We have been quantitative and precise while characterizing these phenomena. Leveraging the interpretability, steerability that our controlled synthetic setup provides, we have access to ground truth and thus we can put exact numbers on the model’s behavior. Evaluation of LLMs is a looming open problem [1], with no existing solution due to the scale, complexity, and lack of accessibility of their huge training corpus. Data leakage is a confounding issue which makes evaluation inherently challenging due to the ambiguity of defining test-train split and similar ambiguity for defining appropriate metrics [2]. Our synthetic setup allows us to bypass both these issues.

In principle, a further study could be performed by training models at scale and confirming at scale, which is what the reviewer is suggesting. We have acknowledged this limitation of our current work in a new section at the end of our introduction. This is a fundamental trade-off when one decides to use synthetic data: we gain interpretability and steerability by having a synthetic DGP where everything is controlled and this allows us to make precise quantitative statements and formulate hypotheses for the origin of phenomena.

Furthermore, we’d like to point the reviewer to a few studies where graph navigation was tested in-context in language models at large-scale:

Mommenejad et al. Evaluating Cognitive Maps in Large Language Models with CogEval: No Emergent Planning (NeurIPS 2023)

Saparov et al. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (ICLR 2023)

Dziri et al. Faith and Fate: Limits of Transformers on Compositionality (NeurIPS 2023)

We have included a figure in our appendix containing example prompts and model outputs from each of these papers.

Moreover, a recent mechanistic study even designs probes to extract the underlying logical graph during stepwise inference in LLMs!:

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models Hou et al (2023) https://arxiv.org/pdf/2310.14491.pdf

We thank you for your positive feedback and we are happy that you think casting stepwise inference as graph navigation is a good choice and that our setting was interesting. We also realized that we might not have motivated the connection between graph navigation and stepwise inference. We have now made it more explicit and also added examples from LLMs (Appendix Figure 8)

Comment: I think my main issue with the paper is that LLMs are trained on very very different data (though in a roughly similar setup). I am not sure the mapping between DAG navigation and what LLMs actually learn is as simple as claimed in the paper as LLMs need to do many other things when trained.

Response: This is a valid concern and we have significantly improved the writeup to address it further. To emphasize, we note that our use of graph navigation was meant as an abstraction of reasoning tasks where stepwise inference protocols have been shown to substantially improve performance. While we did not intend to suggest that graph navigation is a universal model for all such tasks, we do believe that for several tasks currently studied in literature [1,2,3,4], it serves as a faithful abstraction. For example, a LLM is often used to solve problems like the following (taken from [1]): “Imagine a building with 6 rooms. From the lobby you have two choices, you can go to room 1 and room 2. You enter room 1 and at the end of room 1 there is a door that leads to room 3, and room 3 leads to room 5. There’s a chest in room 5. You open it and there is 10 dollars but you do not take the money. Then you exit and start over. This time in the lobby you choose room 2, which has a door to room 4, and room 4 has a door that leads to room 6. You find a chest with 60 dollars in room 6 but you do not take the money. You return to the lobby. You will be able to choose 1 path that leads to the most money. Which room from the lobby will lead to the path where one can make the most money.” Implicitly, the computation required to solve this task involves graph navigation, though the graph is latent – the model has to map natural language onto a graph. This is similar to our setup, where a path sampled from a graph serves the same purpose: it represents a reasoning task that involves achieving a goal (navigation to a node), but the underlying graph is implicit and never fully observed. One can similarly consider the example of math word problems (from GSM8K [5]): “A carnival snack booth made 50 dollars selling popcorn each day. It made three times as much selling cotton candy. For a 5-day activity, the booth has to pay 30 dollars rent and 75 dollars for the cost of the ingredients. How much did the booth earn for 5 days after paying the rent and the cost of ingredients?” Implicitly, the model has to parse these natural language sentences into a computational graph and navigate on it.

Given the similarities highlighted above between our setup and practical tasks, we do believe the claims discussed in our work will transfer to real world settings. However, to perform validation of our generated hypotheses, such as the diversity-accuracy tradeoff and short path bias in stepwise inference, we believe a dedicated study is needed and hence leave it for future work.

Thank you for your detailed review and positive feedback! We are glad that you think our paper presents “an innovative framework”, showcases a “significant advance in understanding logical reasoning paths” and that our study was “comprehensive”. We were also excited to study the diversity-accuracy tradeoff, and believe ours is the first work to systematically and quantitatively study this phenomena. Our goal was exactly this: to devise and explore an appropriate systematic synthetic framework to study the phenomenology of stepwise inference. We address your specific questions below.

Comment: While the paper claims to address stepwise inference …, the experiments seem to focus narrowly on synthetic tasks without sufficient evidence of the model's generalizability to a more realistic or applicable datasets.

Response: Thank you for raising this comment. We have substantially revised our manuscript to address your concern by further motivating the explicit connection between graph navigation and stepwise inference in more realistic settings. Specifically, we have rewritten parts of the introduction (we have elaborated our exposition on the relation between stepwise inference and graph navigation, added a discussion of the limitations of a synthetic data approach), contextualized our results more thoroughly with existing literature, and also edited the formatting and arrangement of figures and results in paper (including promoting the training dynamics figure to the main text since it shows that global planning capability arises after models learn to take correct steps). As an example, our edits now highlight several recent studies that either use or model stepwise inference with LLMs to study tasks that are specific instantiations of our proposed graph navigation problem (e.g., see [1, 2, 3, 4]). We have also included a figure in Appendix (Fig. 9) that describes example prompts and model outputs from each of these papers to exemplify the relation of our proposed synthetic benchmark to realistic setups and tasks considered by prior work in showing the value of stepwise inference. Given these (very recent) studies, we argue the connection between stepwise inference, planning, and graph navigation is explicit and our updated draft ensures this connection is emphasized upon.

We’d also like to contextualize how our methodology of proposing and studying a synthetic problem fits into the literature. Specifically, the goal of our work was to analyze an interesting phenomenon witnessed in LLMs, i.e., stepwise inference, by designing an abstraction of the problem. This enables development of precise mechanistic hypotheses. In LLM and generative model literature, this is a fairly common approach. For example, [5,6] use a synthetically designed setup to develop hypotheses for how ``knowledge” about an entity is stored in pretrained model, show that such knowledge can be manipulated via relatively simple linear transformations. [7] used a procedurally defined synthetic dataset with a relatively small model to demonstrate that emergent abilities seen in neural networks are partially driven by the compositional nature of real world data. In other work [8], use Tracr-compiled transformers to show that if primitive operations involved in a formal algorithm can be implemented in a model, stepwise inference is sufficient for length generalization. Similarly, [9] used context-free grammars to demonstrate that stepwise inference is sufficient to solve problems that require dynamic programming. [10] use CoT to show transformers can learn arbitrary compositions of MLPs in-context. Our work lies in the same regime as these studies and has the goal of defining a well-designed task for studying stepwise inference protocols.

[1] Mommenejad et al. Evaluating Cognitive Maps in Large Language Models with CogEval: No Emergent Planning (NeurIPS 2023)

[2] Saparov et al. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (ICLR 2023)

[3] Dziri et al. Faith and Fate: Limits of Transformers on Compositionality (NeurIPS 2023)

[4] Hou et al. Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models. (arXiv, 2023)

[5] Zeyuan Allen-Zhu and Yuanzhi Li, (2023) Physics of Language Models: Part 3.1, Knowledge Storage (2023)

[6] Zeyuan Allen-Zhu and Yuanzhi Li, Physics of Language Models: Part 3.2, Knowledge Manipulation (2023)

[7] Maya Okawa et al. Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task (NeurIPS 2023)

[8] Zhou et al. What Algorithms can Transformers Learn? A Study in Length Generalization. (arXiv, 2023; https://arxiv.org/abs/2310.16028)

[9] Feng et al, Towards demystifying the mystery behind chain of thought: A theoretical perspective (arXiv, 2023)

[10] Li et al. Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning (NeurIPS 2023)