Deep Fuzzy Multi-view Learning for Reliable Classification

We appreciate the identification of our novelty and the positive comments. Below are our point-by-point responses to your concerns:

Q1: In the Experimental Setup section, the author doesn’t provide details on the noise intensity and noise ratio of the added Gaussian noise.

R1: To create a test set with conflicting instances, following the methodology outlined in (Xu, Cai, et al. "Reliable conflictive multi-view learning."), we apply two transformations: 1)We add Gaussian noise with mean 0 and variance 0.5 to $10\%$ of the instances in the test set. 2)We randomly changed the view information of $50\%$ of the instances in the test set, making the labels of some views inconsistent with the other views. We will clarify this in the next version.

Q2: In the experiments, the authors include some multimodal classification methods, such as PDF and QMF, in the comparison. Is this fair?

R2: Yes, it is fair. To ensure a fair comparison, we replace the backbone networks of QMF and PDF with the same fully connected layer as FUML while preserving their core models and loss functions. These experimental details can be found in Appendix B.3.

Q3: Concerns about the uncertainty estimation results of Handwritten or MSRC.

R3: We compare our FUML with ETMC and ECML on the Handwritten and MSRC datasets for the OOD task. The normal test sets serve as in-distribution, while the conflicting test sets serve as OOD. The evaluation metric is FPR95, with lower values indicating better performance. The results below demonstrate the superiority of FUML.

Q4: The motivation for necessity is unclear, and its impact on performance is unexplained. Why introduce it? Relevant experiments are needed.

R4: In Fuzzy Set Theory, necessity is introduced to quantify the certainty that a sample does not belong to other categories, as membership alone cannot capture between-class relationships. To evaluate the role of necessity, we removed the necessity in FUML and only used conflicts in the fusion process. The experimental results in the table below prove that necessity can’t be removed.

Q5: Can FUML be applied to multimodal classification tasks? If so, what adjustments are needed?

R5: Yes, our FUML framework exhibits strong generalizability and can be readily adapted to various multimodal classification tasks. The primary adjustments required involve replacing the feature extraction backbones to accommodate different data modalities. Specifically, Vision Transformer (ViT) can be employed for image feature extraction, wav2vec for audio processing, TimeSformer for video analysis, and word2vec for textual feature representation.

We appreciate your detailed comments. We believe the following point-to-point response can address all the concerns:

Q1: There are some grammatical errors in the writing.

R1: Thanks. We will correct these grammatical errors you raise and carefully review the manuscript to ensure no other grammatical errors remain.

Q2: Can the ReLU function be replaced by other functions?

R2: No. To map the logits of a neural network as memberships, the process involves two key steps: First, $L_p$-normalization is applied to constrain the logits within the range $[-1, 1]$. Then, a ReLU activation function is employed to further restrict the outputs to the interval $[0, 1]$. The resulting values can be interpreted as memberships for their respective categories. Therefore, both ReLU and $L_p$ normalization are essential parts to qualify membership. If ReLU is replaced by exp or softplus used in the Evidential Deep Learning, the output value will not be in the range of $[0,1]$, which undoubtedly violates the definition of membership.

Q3: In Eq. 8, the exp function is selected as g(). Why not use other monotonically increasing functions, such as ReLU and sigmoid?

R3: The choice of exp is an empirical result: The performance comparison of ReLU, Sigmoid, and the exp function is shown in the following table. From these results, we can find that exp has the best or second-best performance. Therefore, we leverage exp function as g().

Q4: Why not use Euclidean distance or Dot Product Similarity? What is the advantage of cosine similarity?

R4: Compared with Euclidean distance and dot product similarity, cosine similarity only evaluates the consistency of decisions through the angle between memberships, avoiding the sensitivity of Euclidean distance and dot product similarity to vector length, thereby more accurately measuring decision conflicts.

We appreciate your valuable comments. Below is our point-by-point response.

Q1: Weaknesses (1)

R1: 1) In the next version, we will provide a clearer explanation of the theoretical components. 2) An intuitive discussion of the loss function is as follows: Directly aligning category credibility $(m^j_{ik} + 1 - \max\{m_{il}^j \mid l \neq k \})/2$ with the label $y^j_{ik}$ will lead to local optima. Specifically, when $y^j_{ik} = 0$ and $y^j_{il} = 0 (l \neq k)$, minimizing $(1 - \max \\{m_{il}^j \mid l \neq k \\} )$ drives $m_{il}^j$ toward 1 instead of 0, leading to incorrect optimization. To address this, the loss function (Eq.(4)) replaces $(m^j_{ik} + 1 - \max\\{m_{il}^j \mid l \neq k \\})/2$ with $(m^j_{ik} + 1 - m^j_{il})/2$ when $y^j_{ik} = 0$, ensuring correct optimization. We will clarify this in the next version.

Q2: Weaknesses (3)

R2:  Adverse weather conditions and data communication issues often introduce noise and misalignment in multi-view data. Existing EDL-based TMVC methods struggle with accurate classification under such conflicts and may underestimate classification uncertainty. In the next version, we will include this discussion in the Introduction Section.

Q3: Other Comments Or Suggestions (1). 

R3: 1) In medical diagnosis, integrating multiple medical images (e.g., CT, MRI) and genetic data is crucial for disease classification. However, noise or pathological differences between imaging devices would cause inconsistencies, necessitating accurate uncertainty estimation about the classification results for doctors (e.g., ``classified as a malignant tumor, but MRI and CT conflict'') to assist in risk assessment. 2) Our FUML exhibits limited effectiveness in two-view data scenarios. These discussions will be included in the next version.

Q4: Other Comments Or Suggestions (2)

R4: Appendix Tables 7–9 show FUML achieving top or near-top performance in conflicting MVC across eight datasets with diverse noise and misalignment. Notably, FUML significantly outperforms other baselines on unaligned datasets. Overall, FUML demonstrates robustness in handling noisy and unaligned multi-view data, particularly in unaligned cases. This discussion will be incorporated into Appendix C.1 in the next version.

Q5: Questions For Authors (1)

R5: 1) Environmental factors often cause conflicts, which can mislead classification decisions. To mitigate this, we propose fusing only clear (low-uncertainty) and well-aligned (low-conflict) views. Consequently, we employ $(1−u_i^v)(1−o_i^v)$. 2) No, other combinations fail to improve performance. The table below proves this.

Q6: Questions For Authors (2)

R6: Regarding normalization, Appendix C.6 details the impact of the normalization parameter p on performance. The results show that FUML generally achieves optimal performance when p is between 2 and 5. As for activation functions, ReLU is essential for quantizing membership. Using alternative functions (exp or softplus) would yield outputs outside the $[0,1]$, violating fuzzy set theory's membership definition and precluding subsequent category credibility modeling and uncertainty estimation.

Q7: Weaknesses (2) & Questions For Authors (3)

R7: Although the computational complexity is $O(V^2)$, the number of views in multi-view classification (MVC) is typically small ($V < 10$), making the additional computational overhead negligible. Our experiments demonstrate that our method exhibits no significant increase in inference time compared to baselines, approximately by less than $8\\%$ (i.e., 0.0231s vs. 0.0215s).

Q8: Questions For Authors (4)

R8: 1)We add PGD adversarial noise attacks with different maximum perturbation magnitudes (eps) to the test set of the Fashion dataset. The results below show FUML's superior resistance to adversarial noise attacks. 2)We perform an OOD task on the Fashion dataset, using normal test sets as ID and PGD-attacked sets (eps=0.10) as OOD. Evaluated by FPR95 (lower is better), FUML achieved 0.68, outperforming ETMC and ECML (both 1.00).

We sincerely appreciate your valuable feedback. Below are our point-to-point responses:

Q1: Concern 1

R1: In fact, we don't only use Shannon’s entropy for OOD detection. The uncertainty estimation mechanism of FUML includes the following steps: We first calculate the category credibility based on fuzzy membership. The category credibility contains possibility and necessity information, thereby alleviating overconfidence. Then we use Shannon’s entropy of category credibility to estimate uncertainty. The closer the category credibility is to 0.5, the greater the entropy, indicating that the model is less able to make decisions. As can be seen in Figure 8, the uncertainty increases with the increase of noise intensity, which proves the effectiveness of our FUML for OOD detection.

Q2: Concern 2

R2: 1) In fact, rather than overestimating the conflict between two views, our approach correctly reflects subtle semantic differences. Specifically, the difference in lower-ranked class distributions represents the unique characteristics and information captured by each view. For instance, in Figure 1(a), despite both the RGB and text views being classified as “bathroom”, the RGB view includes additional semantics like “toilet paper” and “toilet”, which the text view lacks. 2) We are confused about your intentions. This paper considers multi-view fusion and doesn't take into account the long-tail distribution.

Q3: Concern 3

R3: 1) Our assumption that uncertainty and conflict should jointly determine fusion weights is well-founded: Either higher uncertainty or conflict will reduce its reliability, and it is reliable only if both are low. Specifically, high uncertainty reflects the inherent unreliability of predictions from the corresponding view, while high conflict indicates inconsistencies between decisions from different views, both of which lead to unreliable fusion outcomes. Thus, we use uncertainty and conflict together to compute the fusion weights, so as to more accurately reflect the reliability of the corresponding views. 2) When fusing two uncertainly identical and conflicting views, it is impossible to make a correct decision from a human perspective, and therefore our FUML can't either.

Q4: Concern 4

R4: Although the computational complexity is $O(V^2)$, the number of views is small (usually $V < 10$) in MVC, so the extra computational overhead is negligible. Our experiments indicate that our method exhibits no significant increase in inference time compared to baselines, with an approximate increase of $8\\%$ (i.e., 0.0231s vs. 0.0215s)

Q5: Concern 5

R5: 1) We conduct ablation experiments on the conflicting test sets. The classification accuracy below shows that removing $u_i^v$ or $o_i^v$ leads to performance degradation, indicating the effectiveness of considering both uncertainty and conflict.

2)Since LGG is unavailable, we substitute it with Prokaryotic[1]. The experimental results below demonstrate the superior performance of our FUML. On the NYUD2 dataset with Gaussian noise (mean=0, variance=5), FUML demonstrates superior performance, achieving a classification accuracy of $61.44\pm1.84$ compared to baseline methods (TMC: $59.12\pm1.98$, QMF: $60.32\pm2.63$, PDF: $61.83\pm1.78$).

3)Due to the unavailable source code for "Navigating Conflicting Views: Harnessing Trust for Learning", we compared our FUML with another latest work, TUNED (AAAI 2025)[4], as shown blow, demonstrating the superiority of our method.

Q6: Concern 6

R6: We will correct Eq.(1) and thoroughly proofread the paper for any remaining errors in the next version.

[1]The landscape of microbial phenotypic traits and associated genes.

[2]Trusted Unified Feature-Neighborhood Dynamics for Multi-View Classification.