Edward Meeds

Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension

Gradient Multi-Normalization for Efficient LLM Training

SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training