Niklas Muennighoff

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

OpenThoughts: Data Recipes for Reasoning Models

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality

Humanline: Online Alignment as Perceptual Loss

Crosslingual Reasoning through Test-Time Scaling

HUME: Measuring the Human-Model Performance Gap in Text Embedding Tasks

When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought

UQ: Assessing Language Models on Unsolved Questions

FlexOLMo: Open Language Models for Flexible Data Use

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions

OLMoE: Open Mixture-of-Experts Language Models

Scaling Laws for Precision

RegMix: Data Mixture as Regression for Language Model Pre-training

BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval

Generative Representational Instruction Tuning

ReasonIR: Training Retrievers for Reasoning Tasks

OpenHands: An Open Platform for AI Software Developers as Generalist Agents

MMTEB: Massive Multilingual Text Embedding Benchmark

Language models scale reliably with over-training and on downstream tasks

Bridging the Data Provenance Gap Across Text, Speech, and Video

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?