Gurpreet Gosal

Scaling with Collapse: Efficient and Predictable Training of LLM Families

Power Lines: Scaling laws for weight decay and batch size in LLM pre-training

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs

Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting