Utku Evci

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

Spark Transformer: Reactivating Sparsity in Transformer FFN and Attention

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

Towards Optimal Adapter Placement for Efficient Transfer Learning

Learning Parameter Sharing with Tensor Decompositions and Sparsity