Jonathan Ragan-Kelley

Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping

Ladder Residual: Redefining Tensor Parallelism in Transformers for Accelerated Inference

Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

The Cost of Scaling Down Large Language Models: Reducing Model Size Affects Memory before In-context Learning

How to Guess a Gradient