Antonio Orvieto

Principal Researcher@ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, Tübingen AI Center, Tübingen, Germany·德国·OpenReview

研究方向

Sequence Models · Genomic Language Models · State-Space Models · Theory of Deep Learning · Convex Optimization · Nonconvex Optimization · Deep Learning · Stochastic Optimization

Antonio Orvieto

How does the optimizer implicitly bias the model merging loss landscape?

Selective Rotary Position Embedding

On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach

Design Principles for Sequence Models via Coefficient Dynamics

Improved state mixing in higher-order and block diagonal linear recurrent networks

Revisiting Associative Recall in Modern Recurrent Models

Scaling Behavior of Discrete Diffusion Language Models

Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling

Fixed-Point RNNs: Interpolating from Diagonal to Dense

In Search of Adam’s Secret Sauce

Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture

Generalized Linear Mode Connectivity for Transformers

Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise

When recalling in-context, Transformers are not SSMs

When, Where and Why to Average Weights?

Enhancing Optimizer Stability: Momentum Adaptation of NGN Step-size

Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size

Generalized Interpolating Discrete Diffusion

NIMBA : Towards Robust and Principled Processing of Point Clouds With SSMs