Yuandong Tian

LLM Pretraining with Continuous Concepts

MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes

Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought

Deep Think with Confidence

STEM: SCALING TRANSFORMERS WITH EMBEDDING MODULES

$\mathbf{Li_2}$: A Framework on Dynamics of Feature Emergence and Delayed Generalization

Inpainting-Guided Policy Optimization for Diffusion Large Language Models

SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models

Why RL Updates Look Sparse: An Implicit Compass Drives Optimization Bias

Positional Encoding via Token-Aware Phase Attention

Towards General-Purpose Model-Free Reinforcement Learning

Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization

MagicPIG: LSH Sampling for Efficient LLM Generation

Agent-as-a-Judge: Evaluate Agents with Agents

Composing Global Solutions to Reasoning Tasks via Algebraic Objects in Neural Nets

Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking

Param$\Delta$ for Direct Mixing: Post-Train Large Language Model At Zero Cost

AdvPrefix: An Objective for Nuanced LLM Jailbreaks

GSM-$\infty$: How Do your LLMs Behave over Infinitely Increasing Reasoning Complexity and Context Length?

Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning

Training Large Language Models to Reason in a Continuous Latent Space

AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs

From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications

SpinQuant: LLM Quantization with Learned Rotations

Training Large Language Model to Reason in a Continuous Latent Space

Composing Global Optimizers to Reasoning Tasks via Algebraic Objects in Neural Nets

Agent-as-a-Judge: Evaluating Agents with Agents

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference

Tensor-GaLore: Memory-Efficient Training via Gradient Tensor Decomposition

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge

AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients

From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients

Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planning

The Perfect Blend: Redefining RLHF with Mixture of Judges

Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces

SHARP: Accelerating Language Model Inference by SHaring Adjacent layers with Recovery Parameters