Andrei Semenov

Gradient-Normalized Smoothness for Optimization with Approximate Hessians

Benchmarking Optimizers for Large Language Model Pretraining

Power of Sign: High Probability Bounds Under $(L_0, L_1)$-smoothness and Heavy-Tailed Noise

Just a Simple Transformation is Enough for Data Protection in Split Learning

Clipping Improves Adam and AdaGrad when the Noise Is Heavy-Tailed

Sign Operator for Coping with Heavy-Tailed Noise: High Probability Convergence Bounds with Extensions to Distributed Optimization and Comparison Oracle

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed

Adaptive Regularized Newton Methods with Inexact Hessian