Sosuke Kobayashi

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning

When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars

Efficient Construction of Model Family through Progressive Training Using Model Expansion

Spike No More: Stabilizing the Pre-training of Large Language Models

Spike No More: Stabilizing the Pre-training of Large Language Models