Guoxia Wang

AdaGC: Improving Training Stability for Large Language Model Pretraining

SimReg: Achieving Higher Convergence and Generalization in the LLM Pretraining via Embedding Similarity Regularization

SeWA: Selective Weight Average via Probabilistic Masking

Principal Spectral Regularization Makes Momentum Surpass Adam for LLM Training

FlashMask: Efficient and Rich Mask Extension of FlashAttention