Cong Fang

Strong Correlations Induce Cause Only Predictions in Transformer Training

MVR: Multi-view Video Reward Shaping for Reinforcement Learning

Implicit bias of Hessian Approximation in Regularized Randomized SR1 Method

Preference-Based Process Reward Model for Robust Mathematical Reasoning

Scaling Law for SGD in Quadratically Parameterized Linear Regression

Generalization Analysis of SGD in Linear Regression under Covariate Shift: A View from Preconditioning

Conda: Column-Normalized Adam for Training Large Language Models Faster

AdaPM: a Partial Momentum Algorithm for LLM Training

PaZO: Preconditioned Accelerated Zeroth-Order Optimization for Fine-Tuning LLMs

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads

Learning Curves of Stochastic Gradient Descent in Kernel Regression

SEPARATE: A Simple Low-rank Projection for Gradient Compression in Modern Large-scale Model Training Process