Sanmi Koyejo

Distributional Machine Unlearning via Selective Data Removal

Pretraining Scaling Laws for Generative Evaluations of Language Models

Probing Human Visual Robustness with Neurally-Guided Deep Neural Networks

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms

Who Measures What Matters? An Analysis of Social Impact Evaluations in Foundation Model Reporting

Item Response Scaling Laws: A Measurement Theory Approach to Generalizable Neural Performance Prediction

Efficient Prediction of pass@$k$ Scaling\\in Large Language Models

Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models

AbdCTBench: Learning Clinical Biomarker Representations from Abdominal Surface Geometry

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment for Code

CoDaPO: Confidence and Difficulty-Adaptive Policy Optimization for Language Models

SpecEval: Evaluating Model Adherence to Behavior Specifications

SMARTER SAMPLING FOR LLM JUDGES: RELIABLE EVALUATION ON A BUDGET

Curating High Quality Pretraining Data for Language Models via Compression Ratios

Attention Head Entropy of LLMs Predicts Answer Correctness

UQ: Assessing Language Models on Unsolved Questions

Beyond Mean Scores: Factor Models for Reliable and Efficient AI Evaluation

Evaluating the Robustness of Chinchilla Compute-Optimal Scaling

VeriBench: End-to-End Formal Verification Benchmark for AI Code Generation in Lean 4

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

Data Alignment Predicts Language Model Performance: Evidence from Controlled Experiments in Autoformalization

How Do Large Language Monkeys Get Their Power (Laws)?

Learning to (Learn at Test Time): RNNs with Expressive Hidden States

Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness

Reliable and Efficient Amortized Model-based Evaluation

Language Models May Verbatim Complete Text They Were Not Explicitly Trained On

Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data

Best-of-N Jailbreaking

KGGen: Extracting Knowledge Graphs from Plain Text with Language Models

AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration

The Utility and Complexity of In- and Out-of-Distribution Machine Unlearning

Scaling Laws for Downstream Task Performance in Machine Translation

Reliable and Efficient Amortized Model-based Evaluation

Aligning Compound AI Systems via System-level DPO

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models

Certified Unlearning for Neural Networks

Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs

Learning to (Learn at Test Time): RNNs with Expressive Hidden States

Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?

Incidental Polysemanticity: A New Obstacle for Mechanistic Interpretability

Logits are All We Need to Adapt Closed Models

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF

From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?

Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment

Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs

Nonmyopic Bayesian Optimization in Dynamic Cost Settings

Quantifying Variance in Evaluation Benchmarks

Lean-ing on Quality: How High-Quality Data Beats Diverse Multilingual Data in AutoFormalization

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data

AutoRedTeamer: An Autonomous Red Teaming Agent Against Language Models

MoSH: Modeling Multi-Objective Tradeoffs with Soft and Hard Bounds

Collapse or Thrive: Perils and Promises of Synthetic Data in a Self-Generating World

Attacking Audio Language Models with Best-of-N Jailbreaking