Yutong He

TAH-QUANT: Effective Activation Quantization in Pipeline Parallelism over Slow Network

Mixture-of-Channels: Exploiting Sparse FFNs for Efficient LLMs Pre-Training and Inference

Replacing Multi-Layer Perceptron with Taylor Expansion

MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling

Subspace Optimization for Large Language Models with Convergence Guarantees

Subspace Optimiztion for Large Language Models with Convergence Guarantees

A Mathematics-Inspired Learning-to-Optimize Framework for Decentralized Optimization