Laurence Aitchison

Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers

Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations

How to set AdamW's weight decay as you scale model and dataset size

Function-Space Learning Rates

Residual Stream Analysis with Multi-Layer SAEs

Scale-invariant attention