Srinadh Bhojanapalli

Spark Transformer: Reactivating Sparsity in Transformer FFN and Attention

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count

Scalable In-context Ranking with Generative Models

Spark Transformer: How Many FLOPs is a Token Worth?

Mimetic Initialization Helps State Space Models Learn to Recall

Functional Interpolation for Relative Positions improves Long Context Transformers

Dual-Encoders for Extreme Multi-label Classification

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure