Personalize to generalize: Towards a universal medical multi-modality generalization through personalization

Personalized medicine is a groundbreaking healthcare framework for the $21^{st}$ century, tailoring medical treatments to individuals based on unique clinical characteristics, including diverse medical imaging modalities. These modalities differ significantly due to distinct underlying imaging principles, creating substantial challenges for generalization in multi-modal medical image tasks. Previous methods addressing multi-modal generalization rarely consider personalization, primarily focusing on common anatomical information. This paper aims to connect multi-modal generalization with the concept of personalized medicine. Specifically, we propose a novel approach to derive a tractable form of the underlying personalized invariant representation $\mathbb{X}_h$ using individual-level constraints and a learnable biological prior. We demonstrate that learning a personalized $\mathbb{X}_h$ is both feasible and beneficial, as this representation proves highly generalizable and transferable across various multi-modal medical tasks. Our method is rigorously validated on medical imaging modalities emphasizing both physical structure and functional information, encompassing a range of tasks that require generalization. Extensive experimental results consistently show that our approach significantly improves performance across diverse scenarios, confirming its effectiveness.