Zhiqi Bu

Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate

Deep Progressive Training: scaling up depth capacity of zero-layer model

Adaptive parameter-efficient fine-tuning via multi-task optimization on subset selection

Towards hyperparameter-free optimization with differential privacy

DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction

MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

Gradient descent with generalized Newton’s method

Not all parameters are equal: a Hessian informed differential learning rate for deep learning

Zero redundancy distributed learning with differential privacy

Unlearning as Multi-Task Optimization: a normalized gradient difference approach with adaptive learning rate