Sho Takase

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning

Efficient Construction of Model Family through Progressive Training Using Model Expansion

Scaling Laws for Upcycling Mixture-of-Experts Language Models

Spike No More: Stabilizing the Pre-training of Large Language Models

Spike No More: Stabilizing the Pre-training of Large Language Models