Dianhai Yu

AdaGC: Improving Training Stability for Large Language Model Pretraining

SeWA: Selective Weight Average via Probabilistic Masking

SimReg: Achieving Higher Convergence and Generalization in the LLM Pretraining via Embedding Similarity Regularization

Principal Spectral Regularization Makes Momentum Surpass Adam for LLM Training

FlashMask: Efficient and Rich Mask Extension of FlashAttention