Lei Wu

Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws

How Transformers Get Rich: Approximation and Dynamics Analysis

GradPower: Powering Gradients for Faster Language Model Pre-Training

How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training

How Transformers Implement Induction Heads: Approximation and Optimization Analysis