Ruijun Huang

Metis: Training LLMs with FP4 Quantization

Oracle-MoE: Locality-preserving Routing in the Oracle Space for Memory-constrained Large Language Model Inference

Large Learning Rates without the Agonizing Pain: Dispelling the Curse of Singularities in Deep Neural Networks

Efficiently pre-training language models with mixtures of cluster-oriented, trainability-aware experts