Xihui Lin

A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts

S2-Attention: Hardware-Aware Context Sharding Among Attention Heads