Vaishaal Shankar

OpenThoughts: Data Recipes for Reasoning Models

Datasets, Documents, and Repetitions: The Practicalities of Unequal Data Quality

Language models scale reliably with over-training and on downstream tasks

TiC-LM: A Multi-Year Benchmark for Continual Pretraining of Language Models