Rylan Schaeffer

Pretraining Scaling Laws for Generative Evaluations of Language Models

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms

Efficient Prediction of pass@$k$ Scaling\\in Large Language Models

Item Response Scaling Laws: A Measurement Theory Approach to Generalizable Neural Performance Prediction

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment for Code

Curating High Quality Pretraining Data for Language Models via Compression Ratios

Evaluating the Robustness of Chinchilla Compute-Optimal Scaling

VeriBench: End-to-End Formal Verification Benchmark for AI Code Generation in Lean 4

A $Min-p$ Blueprint for More Rigorous Science in Empirical Machine Learning Research

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

Chain-of-Thought Hijacking

How Do Large Language Monkeys Get Their Power (Laws)?

Best-of-N Jailbreaking

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models

Uncovering Latent Memories in Large Language Models

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World

Incidental Polysemanticity: A New Obstacle for Mechanistic Interpretability

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?

Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment

Quantifying Variance in Evaluation Benchmarks

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data

Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World

Attacking Audio Language Models with Best-of-N Jailbreaking