Dan Alistarh

The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm

Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization

Trion: FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of LLMs

DarwinLM: Evolutionary Structured Pruning of Large Language Model

The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM

Beyond Outliers: A Study of Optimizers Under Quantization

Expand Neurons, Not Parameters

Toy Models of Combinatorial Interpretability

Hogwild! Inference: Parallel LLM Generation via Concurrent Attention

Quartet: Native FP4 Training Can Be Optimal for Large Language Models

Wasserstein Distances, Neuronal Entanglement, and Sparsity

HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs

Unified Scaling Laws for Compressed Representations

LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

EvoPress: Accurate Dynamic Model Compression via Evolutionary Search

Scalable Mechanistic Neural Networks

Efficient Data Selection at Scale via Influence Distillation

EvoPress: Towards Optimal Dynamic Model Compression via Evolutionary Search

Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models

Quantized Optimistic Dual Averaging with Adaptive Layer-wise Compression

Layer-wise Quantization for Quantized Optimistic Dual Averaging

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations

UNAST: Unified framework for Neural Architecture Search for Transformers