Shawn Tan

Distilling to Hybrid Attention Models via KL-Guided Layer Selection

TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments

Do Larger Language Models Generalize Better? A Scaling Law for Implicit Reasoning at Pretraining Time

A Partition Cover Approach to Tokenization

PaTH Attention: Position Encoding via Accumulating Householder Transformations

Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler