A Taxonomy of Transcendence

We thank the reviewer for their time and thoughtful comments, and we provide responses below.

The motivations in the introduction can be better grounded with further citation from previous ML, NLP, or LLM literature, e.g., Line 26, “allow a group to vote” is relevant to the line of research on bagging.

We appreciate the suggestion and will add further citations to the introduction.

Current findings are mostly based on training a pre-trained GPT2 (except Section 7 with Llama 3.2). It would be good to know how model parameters and pre-training design choices will impact the learning and robustness of the skill acquisition (denoising, selection, and generalization). Other newer models with similar sizes (<2B) of GPT2 can also be experimented with, e.g., Phi-1.5, Pythia, Qwen2, and SmolLM.

We agree that comparing different pre-training models in our framework would be interesting. We will add experiments with more model architectures to the final version of the paper.

Although the analysis framework is interesting, current observations regarding diversity and CoT seem common and similar to prior findings from beyond Allen-Zhu’s line of research.

Prior work from Allen-Zhu’s line of research shows that knowledge extraction benefits from phrasing diversity and that CoT is key to knowledge manipulation. We extend these findings by studying the multihop knowledge task in which we show that diversity of the one-hop facts can improve generalization on two-hop knowledge composition. We also connect the compositional abilities to a complexity argument which is distinct from Allen-Zhu’s work. Additionally, we posit that CoT turns an unsupervised generalisation task into an implicit skill-selection problem, which we hope to further formalize in future work. We believe that these observations along with our unifying framework contribute to prior findings and extend them into the new setting of transcendent composition across disjoint experts.

There are many citation problems: line 70, no year. Line 141, what is the wisdom of the crowd? Line 301, use {} over the whole team’s name.

We appreciate your notes on this and will update the citations in the final version of the paper.

Lines 208-210: How is this guaranteed? Is there any probing or pre-training data analysis to show that the model does not include any facts?

We agree that the claim needs explicit evidence. We will add a section in the appendix specifying the data generation process and add a probing analysis to guarantee the model has not seen facts prior to training. To provide more details on the dataset generation process – We start by asking GPT-4o for a set of fictional country names to “seed” the fictional graph. We then run a BFS-like procedure in which for each node, if the node has updated neighbors, those updated neighbors are used as context to generate the fictional name for the current node. This procedure guarantees that GPT-4o has not seen the original entity name when generating its fictional replacement. To further increase diversity, we also provide a starting letter drawn at random for each entity.

Previous work on knowledge conflicts [1][2] shows that there is a ripple effect during knowledge acquisition or editing – one edge can influence the relevant multi-hop neighbors. Can the current evaluation framework provide extended findings considering the ripple effect?

We would also be interested in considering other use cases of our evaluation framework! We plan to release our code to make our framework available to other researchers.

We thank you again for the helpful comments on our paper. We believe that we answered the main drawbacks raised in the review, and would appreciate it if you would raise your score if you believe that your concerns were addressed.

We thank the reviewer for their time and thoughtful comments, and we provide responses below.

The knowledge graph construction uses GPT-4o, which means the "unseen" facts are still artificially likely according to GPT, so the experimental setup is biased.

We appreciate the reviewer’s point. While GPT-4o is indeed used to generate entity names, we emphasize that these fictional names are produced via a carefully structured BFS-like process, conditioned only on a randomly chosen starting letter and previously generated fictional entities. This minimizes bias toward specific factual associations.

Expert coverage is not clearly defined in section 5.

We will clarify the definition of expert coverage in section 5. Essentially, an expert with coverage c over a graph of M edges will have in expectation c*M correct facts in their personal knowledge graph.

The dataset used is cited but no information about it is provided, nor how exactly it is anonymized by GPT-4o, making it difficult to evaluate it.

We will add a section in the appendix specifying the data generation process. The original dataset is a knowledge graph based off of Wikidata. Our anonymization process involves replacing the entity names of this original graph: We start by asking GPT-4o for a set of fictional country names to “seed” the fictional graph. We then run a BFS-like procedure in which for each node, if the node has updated neighbors, those updated neighbors are used as context to generate the fictional name for the current node. This procedure guarantees that GPT-4o has not seen the original entity name when generating its fictional replacement. We also provide a starting letter drawn at random to increase diversity.

The task is somewhat simplistic since it amounts to pure relational knowledge learning + transitive closure. This is very specific but the paper overclaims the generality of the result.

We acknowledge that our task is a simplified setting; however, we believe that this offers a useful model system in which to explore our framework. We encourage future work to explore more general and real world settings and will be careful not to overclaim the generality of our experimental results.

The paper is very repetitive, stating the definition of the taxonomy again in every section without necessarily saying anything new about it.

We appreciate the feedback, and will keep this in mind when editing the final version.

Experiments use the same language model, GPT-2, raising doubt about their generalizability.

We agree that comparing different pre-training models in our framework would strengthen the generalizability of our results. For the skill generalization experiments, we report findings on Llama 3.2. We will add experiments with more model architectures to the final version of the paper.

Theorem 3.1: missing >0

Thank you for catching this! We will correct Theorem 3.1.

We thank you again for the helpful comments on our paper. We believe that we answered the main drawbacks raised in the review, and would appreciate it if you would raise your score if you believe that your concerns were addressed.

We thank the reviewer for their time and thoughtful comments, and we provide responses below.

The main focus of this paper is on how to use an ensemble of language models to perform better than any of the language models individually; however, the paper does not engage with much of the previous work on language model ensembles...

Our work is in part distinguished from this literature by focusing on data properties that allow transcendence (rather than properties or modifications of the learner), but we agree that an overview of work in ensemble learning and multi-task learning would be valuable. We will expand the related work and add these citations.

Even though they are clearly powerful (especially together), both the ensembling approach and the finetuning approach are straightforward and not novel (as far as I can tell), and so I am not sure that many of the formalizations (especially in Section 3.1) are necessary to put in the main paper - in my opinion, they reduce the readability.

We believe the formalization of each setting is necessary, in particular formalizing each setting within the framework provided by [1]. However, we will go through the formalizations and try to make the notation more readable and lightweight if possible.

The discussion is extremely limited, and so the implications for theory and practical applications (and the scope of these) are unclear.

Thank you for the feedback—our discussion section will be expanded to clarify both theory and practice:
Theory: We will add a concise subsection that

• Clarifies how our taxonomy formalizes previously intuitive concepts, providing a framework to analyze the capabilities of an imitative model.
• Discusses how our findings relate to existing classical results in which we (i) link skill-denoising to classical ensemble learning under low-temperature decoding, (ii) formalize skill-selection as implicit expert-routing whenever the input-conditioned source density $g(i \mid x)$ correlates with reward, and (iii) connect our findings on compositional solutions to work on MDL and simplicity bias.

Practice: We will supply concrete take-aways on the importance of data diversity that can allow an imitative model to transcend the sources it is trained on, both for understanding the capabilities of current language models and for future data curation efforts. For instance, curating data to maximize uncorrelated errors and tagging topical metadata so models can automatically learn routing without specialized architectures.

Scope and limitations: We will explicitly state that the taxonomy applies when data can be partitioned into sources with distinct noise or support; tasks that require skill discovery or tool use lie beyond our present scope. We additionally acknowledge that our controlled experimental setup, while valuable for isolating specific phenomena, may differ from real-world settings.

These additions will fit in one page of the camera-ready draft and eliminate ambiguity about implications.

Using GPT-4o to generate names and rephrasings runs a small risk of introducing confounds. For example, the specific process by which the entities were renamed is not stated, but if it used GPT-4o directly (e.g., by asking the model to rename specific entities), there is a possibility that there is some relation between these names and the real names that is also detectable by the GPT-2 models, though this does seem unlikely to have a strong effect.

We will add a section in the appendix specifying the data generation process. We avoid providing GPT-4o with the original entity name for this reason. Specifically, we start by asking GPT-4o for a set of fictional country names to “seed” the fictional graph. We then run a BFS-like procedure in which each node’s updated neighbors are used as context to generate the fictional name for the current node. We also provide a starting letter drawn at random to increase diversity. This guarantees that GPT-4o provides a fictional entity name based only off of its fictional context.

Based on Appendix A1, Theorem 3.1 in the main body of the text is incomplete.

Thank you for catching this! We will correct Theorem 3.1.

Figure 5 is not referenced in the main text.

We will add a reference to Figure 5 in the skill selection experiment section.

We thank you again for the helpful comments on our paper. We believe that we answered the main drawbacks raised in the review, and would appreciate it if you would raise your score if you believe that your concerns were addressed.

We thank the reviewer for their time and thoughtful comments, and we provide responses below.

The broader problem and goal is not defined well enough. The authors talk of ‘inhuman capacity’ (line 20). In this definition, they confuse general human capacity as capacity of the species, and capacities of an individual human, which is what they study. This confusion needs to be clarified, as they do not have a solution to specie-wise inhuman capacities.

Thank you for catching this! This is an important distinction. We define transcendence as exceeding the capacity of the best individual human, but not as exceeding species-level capacity if you allow humans to work together. We will clarify the wording on line 20 and make sure this is clear in other definitions as well.

I would have liked to see, even passingly, a comparison to previous results in ensemble learning and skill denoising, and multi-task learning, and skill generalization, which seem to reach similar conclusions.

Our work is in part distinguished from the literature by our focus on data properties that allow transcendence (rather than properties or modifications of the learner), but we agree that an overview of work in ensemble learning and multi-task learning would be valuable. We will expand the related work. Are there any experiments you’d want to see for comparison?

The generalization results are not clear. What is the level of performance that will allow us to say there has been generalization?

We apologize for the lack of clarity in our generalization results, and will try to improve the clarity here: we define generalization as beating both (i) the best individual expert (0 %) and (ii) the strongest baseline (most common entity) 20% on two-hop queries absent from every expert’s support. In our results we achieve 34% with our standard method, which is 14% above the baseline, and with chain-of-thought we achieve 61%, in which case the gains are highly significant relative to the baseline.

Why this proves generalization: Any non-zero score exceeds what any training source can produce; surpassing 20% shows the model learned compositional structure, not frequency heuristics.

We will add the explicit definition above, making the evidence unambiguous.

We thank you again for the helpful comments on our paper. We believe that we answered the main drawbacks raised in the review, and would appreciate it if you would raise your score if you believe that your concerns were addressed.