Atli Kosson

Weight Decay may matter more than µP for Learning Rate Transfer in Practice

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks