Bettina Messmer

FineWeb2: One Pipeline to Scale Them All — Adapting Pre-Training Data Processing to Every Language

On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists

On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks