Minsik Cho

Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference

EpiCache: Episodic KV Cache Management for Long Conversational Question Answering

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity

SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models

LazyLLM: DYNAMIC TOKEN PRUNING FOR EFFICIENT LONG CONTEXT LLM INFERENCE

SPD: Sync-Point Drop for efficient tensor parallelism of Large Language Models