David Lopez-Paz

Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling

Beyond Multi-Token Prediction: Pretraining LLMs with Future Summaries

Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

The Pitfalls of Memorization: When Memorization Hurts Generalization