Róbert Csordás

GIM: Improved Interpretability for Large Language Models

Decoupling The "What" and "Where" With Polar Coordinate Positional Embedding

Mechanistic evaluation of Transformers and state space models

Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via MoEs

Do Language Models Use Their Depth Efficiently?

Measuring In-Context Computation Complexity via Hidden State Prediction

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models