Depen Morwani

PhD student@Harvard University, Harvard University·美国·OpenReview

研究方向

Deep Learning Optimization · Deep Learning Theory · Inductive Bias of different architectures

6.5

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton

ICLR 2026Poster

通讯

5.0

Seesaw: Accelerating Training by Balancing Batch Size and Learning Rate Scheduling

ICLR 2026Poster

二作

5.0

Adam or Gauss-Newton? — A Comparative Study In Terms of Basis Alignment and SGD Noise

ICLR 2026Rejected

三作

4.0

LOTION: Smoothing the Optimization Landscape for Quantized Training

ICLR 2026Rejected

二作

3.5

Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants

合作者 (20)

Costin-Andrei Oncescu

Depen Morwani

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton

Seesaw: Accelerating Training by Balancing Batch Size and Learning Rate Scheduling

Adam or Gauss-Newton? — A Comparative Study In Terms of Basis Alignment and SGD Noise

LOTION: Smoothing the Optimization Landscape for Quantized Training

Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants

How Does Critical Batch Size Scale in Pre-training?

A New Perspective on Shampoo's Preconditioner

SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling

Deconstructing What Makes a Good Optimizer for Autoregressive Language Models