Andrew M Saxe

Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures

Comparing the learning dynamics of in-context learning and fine-tuning in language models

Distinct Computations Emerge From Compositional Curricula In-Context

Bias-mediated early linearization drives simplicity bias in ReLU networks

Strategy Coopetition Explains the Emergence and Transience of In-Context Learning

Algorithm Development in Neural Networks: Insights from the Streaming Parity Task

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

Training Dynamics of In-Context Learning in Linear Attention

Memory by accident: a theory of learning as a byproduct of network stabilization

From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks

Early learning of the optimal constant solution in neural networks and humans

Make Haste Slowly: A Theory of Emergent Structured Mixed Selectivity in Feature Learning ReLU Networks

A Theory of Initialisation's Impact on Specialisation

When Are Bias-Free ReLU Networks Effectively Linear Networks?