We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including "elegantly bridge the gap", "effectively filter noise and preserve task-relevant information" and "consistently outperforming baselines". Our responses to your concerns are presented in lists as follows. We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.

Q1: About the addressed limitation.

A1: The fourth paragraph outlines several known limitations of prior work, which motivated the design of our method. Below, we explain how CyIN attempt to address each limitation:

• Insufficient exploitation of missing information:
  CyIN incorporates a cyclic cross-modal translation mechanism that allows available modalities to reconstruct missing ones via bottleneck latents, promoting deeper exploitation of cross-modal information. Unlike previous direct imputation method, our model leverages available modalities to reconstruct the missing ones in a compact informative space.

• Task-unrelated noise interruption:
  CyIN introduce IB to filter out irrelevant noise and redundancy by compressing representation. The informative space retains task-relevant information for multimodal representation, which not only reduce the reconstruction difficulty in incomplete multimodal learning, but benefit representation compression in complete multimodal learning.

• Lack of unified modeling for varying missing modality scenarios:
  CyIN can handle arbitrary missing modality patterns without retraining separately by explicitly modeling diverse mising modalities scenarios with cyclic translation in a shared informative latent space, which improves generalization under unknown missing circumstances in real-world.

• Sacrificing complete multimodal performance:
  CyIN consistently maintains state-of-the-art performance for both complete and incomplete modality inputs.

Q2: Theoretical grounding of combining IB with cyclic translation.

A2: We present further explanation for the theoretical analysis of CyIN. The optimization objective for combining IB with cyclic translation is deduced as:

$$\mathcal{L} = \mathcal{L} _ {IB} + \gamma ( \mathcal{L} _ {rec} + \mathcal{L} _ {cyc} )$$

Considering multimodal learning with two modalities without loss of generality, the proposed CyIN can be formulated as follows:

S₁ → B₁ ⇄ B₂ ← S₂
   ↓  ↓
   T₁ T₂

• The left and right side with $S_1 \rightarrow B_1 \rightarrow T_1$ and $S_2 \rightarrow B_2 \rightarrow T_2$ denote the chain of information bottleneck. Note that the source and target states have $S/T\in$ { $F_{S_1},F_{S_2}$ }/{ $F_{T_1},F_{T_2}$ } in the token-level IB or $S\in$ { $F_{S_1},F_{S_2}$ },$T \in y$ in the label-level IB, both of which have been theoretically deduced in our paper, formulated as:

$$\mathcal{L}_{IB} = \underbrace{[I(S _ 1; B _ 1) - \beta I(B _ 1; T _ 1)]} _ {\text{IB for modality 1}} + \underbrace{[I(S _ 2; B _ 2) - \beta I(B _ 2; T _ 2)]} _ {\text{IB for modality 2}}$$

• The middle part $B _ 1 \rightleftharpoons B _ 2$ denotes the cyclic translation with $B _ 1 \rightharpoonup B ^ {rec} _ {2 \rightarrow 1} \rightharpoonup B ^ {cyc} _ 1 \sim B _ 1$ and $B _ 2 \rightharpoonup B ^ {rec} _ {1 \rightarrow 2} \rightharpoonup B ^ {cyc} _ 2 \sim B _ 1$, where $\rightharpoonup$ denotes the translation process while $\sim$ denotes alignment with the original bottleneck. Considering $\Gamma$ as the translator network, the translation process can be divided into cross-modal reconstruction across diverse unimodal latents $B ^ {rec} _ {1 \rightarrow 2}=\Gamma _ {1 \rightarrow 2}(B _ 1)$ and cyclic translation from the reconstructed latent back to the origin modality $B ^ {cyc} _ 2 = \Gamma _ {2 \rightarrow 1}(B ^ {rec} _ {1 \rightarrow 2})$, optimized by:

$$\mathcal{L} _ {rec} = \underbrace{D _ {KL}(B _ 1; B ^ {rec} _ {2 \rightarrow 1})} _ {Reconstruction\ from\ modality\ 1 \rightharpoonup 2} + \underbrace{D _ {KL}(B _ 2; B ^ {rec} _ {1 \rightarrow 2})} _ {Reconstruction\ from\ modality\ 2 \rightharpoonup 1} ,\quad \mathcal{L} _ {cyc} = \underbrace{D _ {KL}(B _ 2; B ^ {cyc} _ 2)} _ {Translating\ Back\ from\ modality\ 2 \rightharpoonup 1} + \underbrace{D _ {KL}(B _ 1; B ^ {cyc} _ 1)} _ {Translating\ Back\ from\ modality\ 1 \rightharpoonup 2}$$

• When both modalities are presented as complete multimodal learning, we can regard the cyclic translation $B_1 \rightleftharpoons B_2$ as sort of cross-modal interaction enhancing modality-shared dynamics extraction. Such cross-modal synergies can benefit the multimodal fusion process $F_M=D_M(B_1,B_2)$ to attain the final discriminative representation.

• When one of the modalities is missing, the framework is required to conduct incomplete multimodal learning. Assuming $u_1$ is missing, we can obtain bottleneck latents $B_2$ from modality $u_2$ to reconstruct the bottleneck latents $B^{rec}_1$, denoted as $B_2 \rightharpoonup B^{rec}_1$, which can be further decoded as the supplementary information for the multimodal fusion process $F_M=D_M(B^{rec}_1,B_2)$.

The aforementioned two-modality scenarios can be generalized to arbitrary modality pairs without constraint, thereby facilitating efficient scaling abilityof CyIN.

Q3: Parameter count, FLOPs, and training time.

A3: We have reported the computational efficiency in Table . Compared with the state-of-the-art methods, the proposed CyIN achieves the lowest total parameter count (123.49M) and FLOPs (1.594T). The inference time of CyIN is $\sim3\times$ faster than GCNet and $\sim5\times$ faster than IMDer.

Due to the cyclic translation process during training, the training time of CyIN is slightly slower than GCNet leveraging Graph Neural Network in modality reconstruction, but still faster then IMDer utilizing diffusion models for reconstruction.

---

Table I. Computational efficiency comparision on MOSI dataset.

---

Q4: Experiments on large-scale or non-affective tasks.

A4: We have extended experiments on 6 non-affective datasets with 2–4 modalities, including:

• Multimodal Recommendation SOTA$^1$ Paper: [1] Jin Li et.al. 2025. Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations. In Proceedings of the ACM on Web Conference 2025 (WWW'25).

• Multimodal Face Anti-spoofing and Dense Prediction SOTA$^2$ Paper: [2] Shicai Wei et.al. 2024. Robust Multimodal Learning via Representation Decoupling. 18th European Conference Computer Vision (ECCV'24).

• Multimodal Medical Segmentation SOTA$^3$ Paper: [3] Pipoli, Vittorio et.al. 2025. IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).

The experimental results shown below illustrate CyIN reaches the state-of-the-art performance, further demonstrating the effectiveness of CyIN in generalizing to diverse mainstream architectures.

The scale of datasets varies from 1.4k samples (NYU v2) to 65k samples (Tiktok), 87k samples (CASIA-SURF), 139k samples (Amazon Baby), which partially show the scaling performance of the proposed CyIN. We leave exploration on larger scale datasets in the future work.

---

Table. Experiment for Multimodal Recommendation task of accuracy and fairness (%) under random missing protocol with 0.4 missing rate as in the SOTA¹ paper.

---

Table. Experiment for Multimodal Face Anti-spoofing task on CASIA_SURF dataset with ACER (↓) under fixed missing protocol as in the original SOTA² paper.

---

Table. Experiment for Multimodal Dense Prediction task on NYU v2 dataset with mIoU (↑) under fixed missing protocol as in the original SOTA² paper.

---

Table VI. Experiment for Multimodal Medical Segmentation task on BraTS2023 dataset with DSC (%) (↑), averaged over four modalities under fixed modality setting in the original SOTA³ paper.

---

Q5: Typo.

A5: Thank you for pointing out. We will carefully check the manuscript.

We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including "unified handling", "clear writing" and "comprehensive experiments". Our responses to your concerns are presented in lists as follows. We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.

Q1: Connection between IB and Cyclic Translation.

A1: Our key contribution lies in the novel integration for IB and cyclic translation within a unified framework designed for both complete and incomplete multimodal learning.

Specifically, the IB serves as an effective representation learning principle by encouraging compact and task-relevant latent spaces, which helps reduce the complexity of high-dimensional inputs while preserving discriminative modality-specific and -shared information. Cyclic translation, on the other hand, enhances cross-modal reconstruction by reinforcing consistency and structural alignment between modalities. It captures modality-shared features, facilitating robust reconstruction when some modalities are missing.

We have increased a theoretical analysis in combining these two approaches in our work as follows. Considering multimodal learning with two modalities without loss of generality, the proposed CyIN can be formulated as follows:

S₁ → B₁ ⇄ B₂ ← S₂
   ↓  ↓
   T₁ T₂

Combining IB with cyclic translation, we can deduce the theoretical loss as follows,

$$\mathcal{L} = \mathcal{L} _ {IB} + \gamma (\mathcal{L} _ {rec} + \mathcal{L} _ {cyc})$$ $$\mathcal{L} = \underbrace{[I(S _ 1; B _ 1) - \beta I(B _ 1; T _ 1)]} _ {\text{IB for modality 1}} + \underbrace{[I(S _ 2; B _ 2) - \beta I(B _ 2; T _ 2)]} _ {\text{IB for modality 2}} + \underbrace{[D _ {KL}(B _ 1; B ^ {rec} _ {2 \rightarrow 1}) + D _ {KL}(B _ 2; B ^ {rec} _ {1 \rightarrow 2})]} _ {\text{Reconstruction from modality 1} \rightleftharpoons \text{2}} + \underbrace{[D _ {KL}(B _ 2; B ^ {cyc} _ 2) + D _ {KL}(B _ 1; B ^ {cyc} _ 1)]} _ {\text{Translating Back from modality 2} \rightleftharpoons \text{1}}$$

Minimizing above reconstruction loss, we can obtain informative unimodal bottleneck latents $B _ u$ for each modality $u$, which contains inter-modal features to conduct productive multimodal fusion. Comparing with directly fusing unimodal representations $F _ u$, fusion process implemented on the bottleneck latents $B _ u$, denoted as $F _ M = D _ M(B _ 1, B _ 2)$, can benefit from the compression ability of information bottleneck and lead to less reconstruction difficulty in cross-modal translation.

• When both modalities $u _ 1$ and $u _ 2$ are presented, known as complete multimodal learning, we can regard the cyclic translation $B _ 1 \rightleftharpoons B _ 2$ as sort of cross-modal interaction enhancing the extraction of modality-shared dynamics. Such cross-modal synergies can benefit the multimodal fusion process $F _ M = D _ M(B _ 1, B _ 2)$ to attain the final discriminative representation.

• When one of the modalities is missing, the framework is required to conduct incomplete multimodal learning. Assuming $u _ 1$ is missing, we can obtain bottleneck latents $B _ 2$ from modality $u _ 2$ to reconstruct the bottleneck latents $B ^ {rec} _ 1$, denoted as $B _ 2 \rightharpoonup B ^ {rec} _ 1$, which can be further decoded as the supplementary information for the multimodal fusion process $F _ M = D _ M(B ^ {rec} _ 1, B _ 2)$.

In summary, IB provides compact, robust latent spaces for both understanding and generation, while cyclic translation enhances modality interaction and recovery. Their combination allows CyIN to unify complete and incomplete modality scenarios in the same framework. We believe this integration is not only novel but also essential for scalable and flexible multimodal learning.

Q2: Cost for two-stage training and different modules.

A2: To address this concern, we have quantified the additional computational costs introduced by the two-stage training, VIB networks, multiple translators, and multimodal fusion (multi-head cross-attention) in Table I across GCNet, IMDer, and our proposed CyIN.

---

Table I. Comparison on model complexity for the state-of-the-art methods and the proposed CyIN, including the model parameter (M) and proportion percentage (Pc).

---

CyIN achieves the lowest total parameter count (123.49M) and significantly lower inference time (22.41s) compared to GCNet (70.62s) and IMDer (103.75s). Besides, CyIN costs $17.4\ \mathrm{s/epoch}$ for Stage 1, $213.9\ \mathrm{s/epoch}$ for Stage 2, where Stage 1 for training informative space is relatively fast.

The added parameters and inference latency from the increased modular components as VIB and Translator modules are relatively minor. Specifically:

• VIB introduces only 1.74M parameters (1.41% of the total, or 12.42% of increased parameters), and 0.37 s latency

• Translator adds 1.45M (1.18% of the total, or 10.35% of increased parameters), and 17.51 s latency (Reconstruction for all missing modalities scenarios, 2.92s for each missing scenario)

• Fusion module accounts for the majority increased parameters (9.64M, or 68.80% of increased parameters), and 0.26 s inference latency.

These results demonstrate that the proposed modular extensions are lightweight relative to the overall architecture. Most of the additional cost arises from the fusion mechanism, not from the cyclic translators or VIB modules. Despite these additions, CyIN remains the most efficient in terms of both parameter size and inference latency. We will explore faster translation way like one-step flow matching or consistency model to further speed up the translation process in the future.

We would like to express our sincere gratitude for your thoughtful questions and valuable feedback on our strengths, including "well written paper", "sound technique details", and "convincing experimental results". Our responses to your concerns are presented in lists as follows. We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.

Q1: About novelty and contribution.

A1: We appreciate the reviewer’s feedback and would like to clarify and reaffirm the key contributions of our work to the multimodal learning community:

• Addressing the Limitations of Prior Work: Many existing approaches either neglect the performance degradation caused by real-world challenges such as sensor failure and corrupted inputs, or suffer from task-irrelevant noise when reconstructing them. Moreover, strategies such as data augmentation or imputation often reduce performance on complete multimodal inputs and demand prior knowledge about the missing situation, making it difficult to jointly handle complete and incomplete multimodal scenarios in real-world applications.

• A Unified Framework for Complete and Incomplete Multimodal Learning: Our proposed CyIN framework bridges this gap by unifying complete and incomplete multimodal learning within a shared informative latent space, offering a new direction for building robust multimodal systems capable of handling diverse scenarios without requiring separate models or heuristics. The token- and label-level IB with cyclic translation mechanism over informative latents improves cross-modal interaction in multimodal fusion and enables effective reconstruction of missing modalities.

• Extensive Empirical Experiment Validation: We have validated CyIN across 6 sub-tasks and 10 diverse multimodal datasets involving 2–4 modalities. The framework consistently demonstrates strong performance under various missing modality settings.

Q2: Well-suited for academic exploration but not scalable for solving real-world AI tasks.

A2: We'd like to provide an open-minded discussion of both the academic and application value of our work. Admittedly, VAE and IB are often seen as "one-size-fits-all" solutions, as you suggested. However, academic exploration continues to yield valuable insights, and these models still have broad applicability. For example:

• VAEs are widely adopted in image latent encoding for recent AIGC applications such as Stable Diffusion [4].
• IB is actively being explored for enhancing safety in large language models [5].

Thus, we believe deeper theoretical exploration of VAE and IB in the context of multimodal models is equally important as pursuing application-focused research.

Moreover, to further validate the applicability of our proposed method, we conduct extensive experiments on current industrial multimodal applications, including:

• Sentiment analysis: MOSI, MOSEI datasets
• Emotion recognition: IEMOCAP, MELD datasets
• Multimodal recommendation [1]: Amazon Baby, Tiktok, Allrecipes datasets
• Face anti-spoofing [2]: CASIA-SURF dataset
• Dense prediction [2]: NYUv2 dataset
• Medical segmentation [3]: BraTS2023 dataset

---

Table. Experiment comparison for Multimodal Recommendation task of accuracy and fairness performance (%) on three datasets, including Amazon Baby, Tiktok, and Allrecipes. The modality setting follows the random missing protocol with 0.4 missing rate as in the SOTA¹ [1] paper.

---

---

Table. Experiment comparison for Multimodal Face Anti-spoofing task with Average Classification Error Rate (ACER ↓) on the CASIA-SURF dataset. The modality setting follows the fixed missing protocol as in the original SOTA² [2] paper.

---

---

Table. Experiment comparison for Multimodal Dense Prediction task with mIoU (↑) on the NYUv2 dataset. The modality setting follows the fixed missing protocol as in the original SOTA² [2] paper.

---

---

Table. Experiment comparison for Multimodal Medical Segmentation task with Dice Similarity Coefficient (DSC %) (↑) on the BraTS2023 dataset. The results are averaged over four modalities: FLAIR, T1, T1c, and T2, following the fixed modality setting in the original SOTA³ [3] paper.

---

The experimental results shown above illustrate that CyIN achieves state-of-the-art performance. Besides, we'd like to point out that the training procedure is flexible in different training architecture, including: Transformer-based (Ours), Graph-based (SOTA¹ [1]), CNN-based (SOTA² [2]), Mamba-based (SOTA³ [3]) for both modality-specific feature extraction and multimodal fusion.

Lastly, we scale up CyIN with different sizes of PLMs including BERT, RoBERTa, and DeBERTa-V3 as follows, indicating efficient scaling ability similar to the scaling law of PLM training.

---

Table. Comparison of the proposed CyIN using different sizes of language models on MOSI dataset under both complete and randomly incomplete multimodal settings.

---

In summary, we believe our work provides a new perspective to the multimodal learning community in bridging complete and incomplete multimodal learning.

We sincerely hope that above responses could address your concerns and provide another perspective for the application of the proposed method.

[1] Jin Li et.al. 2025. Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations. In Proceedings of the ACM on Web Conference 2025 (WWW'25).

[2] Shicai Wei et.al. 2024. Robust Multimodal Learning via Representation Decoupling. 18th European Conference Computer Vision (ECCV'24).

[3] Pipoli, Vittorio et.al. 2025. IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).

[4] Rombach R et al. High-resolution image synthesis with latent diffusion models[C] Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR'2022).

[5] Liu Z et al. Protecting your llms with information bottleneck[J]. Advances in Neural Information Processing Systems, 2024 (NIPS'2024).

We would like to express our sincere gratitude for your thoughtful questions and valuable feedback of our strengths including "novel framework" and "enhancing robustness". Our responses to your concerns are presented in lists as follows. We are eager to engage in further discussions with you to address your concerns and enhance the quality of our work.

Q1: About imbalance contribution of diverse modalities.

A1: In our work, we assumed equal contribution across modalities to ensure generality and stability, particularly under randomly missing modality scenarios. However, we acknowledge that in practice, different modalities may contribute unequally depending on the task [4], summarized as balanced multimodal learning [5], another issue in the field of multimodal learning.

To partially address this, we increase experiments by varying the number of cross-moda translator layers in Table I. We observe that allocating more layers in reconstruction allows the model to learn more semantics representation, where the results show more important contribution of text modality than others.

We plan to explore adaptive balancing strategies as part of our future work as stated in Section Limitations.

---

Table I. Performance comparison of CyIN using different CRA layer ratios for cross-modal translation ($la$:$lv$:$av$) under random missing rate $MR=0.7$ on the MOSI dataset.

---

Q2: About computational efficiency and scalability with modality number and data dimension.

A2: As shown in Table II, CyIN achieves the best inference efficiency and lowest FLOPs when handling three input modalitie, comparing with the state-of-the-art methods such as GCNet and IMDer.

---

Table II. Computational efficiency for the state-of-the-art methods and the proposed CyIN on MOSI dataset.

---

Moreover, CyIN is designed to be scalable as the number of modalities increases. The cross-modal translation mechanisms introduce only a minimal computational overhead relative to prior methods (Each cross-modal translator only increase 1.45 M parameters, $\sim 10$% of the total model parameters). This is due to the modular nature of the architecture and the efficient reconstruction performance in the informative space. Since reconstruction is performed in this compressed latent space, The computational cost remains manageable in scenarios with high-dimensional data, where IB serves as a compact and informative representation extractor [6,7].

To further validate scalability, we conducted additional experiments on datasets involving 2 to 4 modalities, as detailed in the next response. These results demonstrate the robustness and efficiency of CyIN across varying modality configurations.

Q3: About experimental verification for broader multimodal tasks.

A3: To further validate the applicability of the proposed method, we conduct extensive experiments on current industrial multimodal applications including:

• Multimodal Recommendation SOTA¹ (3 modalities: visual, audio, textual) paper: [1] Jin Li et.al. 2025. Generating with Fairness: A Modality-Diffused Counterfactual Framework for Incomplete Multimodal Recommendations. In Proceedings of the ACM on Web Conference 2025 (WWW'25).

• Multimodal Face Anti-spoofing (3 modalities: RGB, Depth, IR) and Dense Prediction (2 modalities: RGB, Depth) SOTA² paper: [2] Shicai Wei et.al. 2024. Robust Multimodal Learning via Representation Decoupling. 18th European Conference Computer Vision (ECCV'24).

• Multimodal Medical Segmentation (4 modalities: FLAIR, T1, T1c, T2) SOTA³ paper: [3] Pipoli, Vittorio et.al. 2025. IM-Fuse: A Mamba-based Fusion Block for Brain Tumor Segmentation with Incomplete Modalities. 28th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI'25).

The experimental results shown below illustrate CyIN reaches the state-of-the-art performance.

---

Table III. Experiment comparison for Multimodal Recommendation task of accuracy and fairness performance (%) on three datasets, including Amazon Baby, Tiktok, and Allrecipes. The modality setting follows the random missing protocol with 0.4 missing rate as in the SOTA¹ paper.

---

Table IV. Experiment comparison for Multimodal Face Anti-spoofing task with Average Classification Error Rate (ACER ↓) on the CASIA-SURF dataset. The modality setting follows the fixed missing protocol as in the original SOTA² paper.

---

Table V. Experiment comparison for Multimodal Dense Prediction task with mIoU (↑) on the NYUv2 dataset. The modality setting follows the fixed missing protocol as in the original SOTA² paper.

---

Table VI. Experiment comparison for Multimodal Medical Segmentation task with Dice Similarity Coefficient (DSC %) (↑) on the BraTS2023 dataset. The results are averaged over four modalities: FLAIR, T1, T1c, and T2, following the fixed modality setting in the original SOTA³ paper.

---

Q4: About hyper-parameter sensitivity.

A4: We have conducted additional experiments presented in Table VII, which systematically evaluate the effect of varying the most important hyper-parameters $\beta$ and $\gamma$. The former controls the trade-off degree of mutual information in IB and the latter adjusts the contribution of translation loss.

---

Table VII. Hyper-parameter sensitivity on $\beta$ and $\gamma$ of the proposed CyIN on MOSI dataset with incomplete multimodal learning of random missing rate $MR=0.7$.

---

• Setting $\beta=8-32$ gives better performance as a balanced trade-off for mututal information between $I(S;B)$ and $I(B;T)$. Very small or large values of $\beta$ lead to clear degradation in all metrics, showing that improper bottleneck strength harms performance.

• Setting $\gamma=10$ yields the best overall performance across metrics. A small $\gamma$ results in weak regularization, making it hard to align modalities under severe missing conditions while a large $\gamma$ hurts performance greatly dur to neglecting building effective informative space.

[4] Peng X, Wei Y, Deng A, et al. Balanced multimodal learning via on-the-fly gradient modulation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) 2022.

[5] Zhang H, Wang W, Yu T. Towards robust multimodal sentiment analysis with incomplete data[J]. Advances in Neural Information Processing Systems (NeurIPS) 2024.

[6] Shwartz-Ziv R, Tishby N. Opening the black box of deep neural networks via information. arXiv:1703.00810, 2017.

[7] Saxe A M, Bansal Y, Dapello J, et al. On the Information Bottleneck Theory of Deep Learning. International Conference on Learning Representations (ICLR) 2018.