Jared Tanner

Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers

Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Transformers

DEEP NEURAL NETWORK INITIALIZATION WITH SPARSITY INDUCING ACTIVATIONS

Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes

Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs