Ryo Karakida

Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians

Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation

Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs

On the Parameterization of Second-Order Optimization Effective towards the Infinite Width

Understanding MLP-Mixer as a wide and sparse MLP

SEFAR: SparsE-FeAture-based Regularization for Fine-Tuning on Limited Downstream Data