Naoki Nishikawa

Inference-time Alignment with Rewards in Anisotropic Besov Spaces: Superiority of Neural Networks over Linear Estimators

Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency

Nonlinear transformers can perform inference-time feature learning

From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers

When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars

State Space Models are Provably Comparable to Transformers in Dynamic Token Selection

Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning