Ziyu Zhao

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns

PrunedLoRA: Robust Gradient-Based Structured Pruning for Low-rank Adaptation in Fine-tuning

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging

REMEDY: Recipe Merging Dynamics in Large Vision-Language Models

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering