Shikai Qiu

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks

How to Scale Second-Order Optimization

Customizing the Inductive Biases of Softmax Attention using Structured Matrices

Pathologies of Out-of-Distribution Detection

Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices

Efficient Transfer Learning from Arbitrary Pre-Trained Models