Kwangjun Ahn

General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training

The Belief State Transformer

Does SGD really happen in tiny subspaces?

Linear attention is (maybe) all you need (to understand Transformer optimization)

Adam with model exponential moving average is effective for nonconvex optimization