Kaifeng Lyu

Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice

Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining

The Information Bottleneck of Chain-of-Thought and How Latent CoT Overcomes It

LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?

When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs

Fine-tuning VLMs Without Forgetting Is Easier Than You Think

Safety Alignment Should be Made More Than Just a Few Tokens Deep

Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition

Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks

Efficient stagewise pretraining via progressive subnetworks

Weak-to-Strong Generalization Even in Random Feature Networks, Provably

Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

How Far Are We from Optimal Reasoning Efficiency?

RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval

Shift is Good: Mismatched Data Mixing Improves Test Performance

AI-Assisted Generation of Difficult Math Questions