Yuanzhi Li

Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

Mixture of Parrots: Experts improve memorization more than reasoning

Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems

Physics of Language Models: Part 3.2, Knowledge Manipulation

Interpretability of Language Models for Learning Hierarchical Structures

On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics

Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data

Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process

Understand Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics

Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts

Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs

VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?

SmartPlay : A Benchmark for LLMs as Intelligent Agents

Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP

Knowledge Storage and Extraction in Language Models (Part A)

AgentKit: Structured LLM Reasoning with Dynamic Graphs

Textbooks Are All You Need

Simple mechanisms for representing, indexing and manipulating concepts

Knowledge Manipulation in Language Models (Part B)

How Language Models Learn Context-Free Grammars

Why Clean Generalization and Robust Overfitting Both Happen in Adversarial Training

Positional Description Matters for Transformers Arithmetic

How does overparametrization affect features?

TinyStories: How Small Can Language Models Be and Still Speak Coherent English