Mingze Wang

On The Surprising Effectiveness of a Single Global Merging in Decentralized Learning

Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws

How Transformers Get Rich: Approximation and Dynamics Analysis

GradPower: Powering Gradients for Faster Language Model Pre-Training

Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward

On the Expressive Power of Mixture-of-Experts for Structured Complex Tasks

Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training

How Transformers Implement Induction Heads: Approximation and Optimization Analysis