Huan Li

Conda: Column-Normalized Adam for Training Large Language Models Faster

On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm