Ziwei He

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models

Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs

PonderLM: Pretraining Language Models to Ponder in Continuous Space

Pretraining LLM with Latent Thoughts in Continuous Space

FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension

Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization