Jianqiao Lu

FlexLinearAttention: Compiling a Unified Abstraction into Scalable Kernels for Linear Attention

Enhancing linear attention with residual learning

HSA: Head-wise Sparse Attention for Efficient and Accurate Long-context Inference

DeltaFormer: Unlock the state space of Transformer

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

Model Merging in Pre-training of Large Language Models

FormalAlign: Automated Alignment Evaluation for Autoformalization

UNComp: Uncertainty-Aware Long-context Compressor for Efficient Large Language Model Inference

Process-Driven Autoformalization in Lean 4

UncertaintyRAG: Span Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation

MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation

Scaling Laws For Mixed Quantization In Large Language Models