Zhenyu Zhang

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts

Mixture of Hidden-Dimensions: Not All Hidden-States’ Dimensions are Needed in Transformer

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging

HFT: Half Fine-Tuning for Large Language Models