Yiyuan Ma

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models

Balanced Actor Initialization: Stable RLHF Training of Distillation Reasoning Models

HSA: Head-wise Sparse Attention for Efficient and Accurate Long-context Inference

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

Model Merging in Pre-training of Large Language Models