Junxuan Wang

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition

Attention Layers Add Into Low-Dimensional Residual Subspaces

Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures

Automatically Identifying and Interpreting Sparse Circuits with Hierarchical Tracing