Shengding Hu

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Stuffed Mamba: Oversized States Lead to the Inability to Forget

Stuffed Mamba: State Collapse and State Capacity of RNN-Based Long-Context Modeling

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

Unified View of Grokking, Double Descent and Emergent Abilities: A Comprehensive Study on Algorithm Task

Predicting Emergent Abilities with Infinite Resolution Evaluation