Zhengzhong Liu

Esoteric Language Models

CLEAR: A Cost-Aware Routing System for Edge-Cloud Language Model Collaborative Inference

GQA-$\mu$P: The Maximal Parameterization Update for Grouped Query Attention and Fully Sharded Data Parallel

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing

Faster Video Diffusion with Trainable Sparse Attention

MegaMath: Pushing the Limits of Open Math Corpora

Fast Video Generation with Sliding Tile Attention

Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models

How does controllability emerge in language models during pretraining?

Scaling Long Context Training Data by Long-Distance Referrals

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing