Locally Optimal Descent for Adaptive Stepsize Scheduling

We introduce a dynamic stepsize scheduler based on the local optimal step size, it attains optimal rate in the smooth (non-convex) setting and empirically shown to perform on par with manually fine-tuned schedulers.