Peter Richtárik

Full Professor@King Abdullah University of Science and Technology (KAUST)·沙特阿拉伯·OpenReview

研究方向

model compression · randomized second order methods · federated learning · distributed optimization · randomized algorithms · supervised machine learning · stochastic gradient descent · randomized coordinate descent · optimization

Peter Richtárik

Better LMO-based Momentum Methods with Second-Order Information

Error Feedback for Muon and Friends

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity

Local SGD and Federated Averaging Through the Lens of Time Complexity

Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes

Tighter Performance Theory of FedExProx

Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression

From Muon to Gluon: Bridging Theory and Practice of LMO-based Optimizers for LLMs

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy

Sharper Characterization of the Global Maximizers in Bilinear Programming with Applications to Asynchronous Gradient Descent

On the Convergence of FedProx with Extrapolation and Inexact Prox

BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Training

Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning

Drop-Muon: Update Less, Converge Faster

Symmetric Pruning for Large Language Models

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation

MARINA-P: Superior Performance in Nonsmooth Federated Optimization with Adaptive Stepsizes

A Novel Unified Parametric Assumption for Nonconvex Optimization

Provably Convergent and Private Distributed Optimization via Smoothed Normalization

Supra-Tuning: Combining Outlier and Low-Rank Adaptation for Sparse and Efficient LLM Fine-Tuning

Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression

Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits

MAST: model-agnostic sparsified training

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization

Local Curvature Descent: Squeezing More Curvature out of Standard and Polyak Gradient Descent

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity

ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning

On the Convergence of FedProx with Extrapolation and Inexact Prox

Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy

Smoothed Normalization for Efficient Distributed Private Optimization

Communication-efficient Algorithms Under Generalized Smoothness Assumptions

RAC-LoRA: A Theoretical Optimization Framework for Low-Rank Adaptation

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity

Error Feedback for Smooth and Nonsmooth Convex Optimization with Constant, Decreasing and Polyak Stepsizes

Tighter Performance Theory of FedExProx

A Unified Theory of Stochastic Proximal Point Methods without Smoothness

Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes

Local Curvature Descent: Squeezing More Curvature out of Standard and Polyak Gradient Descent

Prune at the Clients, Not the Server: Accelerated Sparse Training in Federated Learning

MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times

Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning

FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models