Xun Zhou

Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models

Balanced Actor Initialization: Stable RLHF Training of Distillation Reasoning Models

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models

Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts

Hyper-Connections

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods