Kun Yuan

Assistant Professor@Peking University·中国·OpenReview

研究方向

distributed optimization · large-scale optimization · stochastic optimization · efficient training methods in deep learning · distributed deep learning

Kun Yuan

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure

Scaling up Multi-Turn Off-Policy RL and Multi-Agent Tree Search for LLM Step-Provers

TAH-QUANT: Effective Activation Quantization in Pipeline Parallelism over Slow Network

Mixture-of-Channels: Exploiting Sparse FFNs for Efficient LLMs Pre-Training and Inference

Improving MoE Performance and Efficiency with Plug-and-Play Intra-Layer Specialization and Cross-Layer Coupling Losses

Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression

MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling

A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models

Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads

Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold

Subspace Optimization for Large Language Models with Convergence Guarantees

Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance

Achieving Linear Speedup and Near-Optimal Complexity for Decentralized Optimization over Row-stochastic Networks

Subspace Optimiztion for Large Language Models with Convergence Guarantees

A Mathematics-Inspired Learning-to-Optimize Framework for Decentralized Optimization

Achieving Optimal Complexity in Decentralized Learning over Row-Stochastic Networks