Automatically Identify and Rectify: Robust Deep Contrastive Multi-view Clustering in Noisy Scenarios

Explanation for projector and classifier: Thanks. We perform feature mapping and transformation in the latent space using a projector, and the sample predictions are obtained through a classifier. The projector and classifier are shared across different views. We will include the corresponding description in the final version.

Limitations & Future directions of AIRMVC: Thanks. In AIRMVC, we made an initial attempt to identify and rectify noise in an unsupervised setting and designed a robust contrastive learning method to further enhance the robustness of the model. However, the correction process relies heavily on the accuracy of the predicted distribution, which is the primary limitation of AIRMVC. In the future, improving the accuracy of the predicted distribution and exploring other reliable supervisory signals will be promising research directions.

Large-scale motivation experiments: Thanks. Following your suggestions, we conducted experiments on the Caltech101 and STL10 datasets with a 10% noise ratio. The experimental results are presented in Tab.1 and Tab.2. From these results, we observe the same conclusions as those described in the submitted version, presented as follows:

1. Simply merging noisy multi-view data results in the most degraded clustering performance. This is primarily due to the absence of a noise rectification mechanism, which causes the negative effects of noisy views to compound each other. Moreover, the fusion process intensifies the influence of noise, leading to a scenario where multi-view clustering performs even worse than using a single view alone.

2. In comparison to directly correcting noise based on the first view, our proposed AIRMVC demonstrates superior performance. This advantage arises because direct correction from a single view tends to enforce uniformity across views, potentially suppressing essential complementary information. In contrast, our noise detection and rectification strategy effectively removes noisy samples from each view while preserving beneficial cross-view diversity, thereby enhancing the overall clustering performance.

Tab.1 Motivation experiments on STL10 dataset.

Tab.2 Motivation experiments on Caltech101 dataset.

Code: Thanks. Following your suggestion, we will release the code in the final version.

Notation & Typos: Thanks. We will add the notations of Eq.10 and correct the typos in page 7. Furthermore, we will review the entire paper to enhance the overall presentation.

Additional experiments: Thanks. Following your suggestions, with NVIDIA A6000 GPU we conduct experiments on CIFAR10 dataset, which contains 60,000 samples, 4 views and 10 classes. Besides, YouTube is a comprehensive video platform. We extract facial images from videos as a real-world data source. These multi-view facial images may include low-quality samples, which are treated as noise in our analysis. The Youtube dataset comprises 38,654 samples, 4 views and 10 classes. Detailed statistic information of the datasets is demonstrated in Tab.1. From the results shown in Tab.2 and Tab.3, we conclude that AIRMVC could achieve reliable performance on both large-scale dataset and real-world dataset, demonstrating its generalization capability.

Tab.1 Statistic information of the datasets.

Tab.2 Experiment on YouTube dataset. OOM denotes out-of-memory during training process.

Tab.3 Experiment on CIFAR dataset.

Discussion with MVCAN: Thanks. We discuss AIRMVC with MVCAN from three key perspectives:

1. Optimization Strategy: MVCAN adopts a two-level iterative optimization framework, consisting of T-level and R-level optimization to refine the network. In contrast, In contrast, AIRMVC focuses on noise detection and correction using a Gaussian Mixture Model (GMM) and directly optimizes the network.

2. Soft Label Acquisition: MVCAN employs a parameter-decoupled model to obtain view-specific representations and soft labels, mitigating the influence of noisy views. AIRMVC leverages a GMM trained with a shared projector and classifier to generate soft labels.

3. Module Design: MVCAN incorporates unshared parameters, distinct clustering optimization functions, and a two-level iterative optimization approach. In comparison, AIRMVC introduces a dedicated noise detection and correction mechanism, along with a noise-robust contrastive learning framework to enhance model robustness.

Novelty of AIRMVC: Thanks. The novelty of AIRMVC mainly contains the following perspectives.

1. Leveraging GMM, we reformulate the noise identification as an anomaly identification problem and propose a hybrid rectification strategy to automatically correct the noisy data.

2. We design a noise-robust contrastive mechanism to generate more reliable representations. Theoretically, we have demonstrated that the features generated by this mechanism are more beneficial for downstream tasks.

3. Extensive experiments on different benchmark datasets to verify the effectiveness and robustness of AIRMVC.

Explanation for adding noise: Thanks. Different with the noisy alignment, we simulate noisy scenarios by injecting standard Gaussian noise to the original views, excluding the first view. Specifically, we generate random Gaussian noise with the same shape as the view and inject it into the original views at a ratio of x%. The parameter x% scales the generated Gaussian noise, thereby simulating different levels of noise contamination.

Experiments on clean data: Thanks. Following your suggestion, we conduct experiments on clean data with six datasets. The results are shown in Tab.1. Due to character limitations, more results can be found in Tab.1 at https://anonymous.4open.science/r/Res-11B2. From the results, we find that AIRMVC achieves promising performance in both clean and noisy scenarios.

Tab.1 Clean data performance

Explanation for assumption: Thanks. We provide explanation from three perspectives.

1. Different with noisy data align, AIRMVC extends the definition of noisy by considering the presence of noise within views. Since we propose a method for detecting and correcting noise, an "ideal" view is required as a reference standard. In the unsupervised multi-view clustering scenario, there is no available label information. Therefore, we assume that the first view serves as an ideal view, acting as pseudo-supervision to correct noise in the other views.

2. We show this assumption with a real-world multi-view scenario, i.e., the ideal view supplements and corrects the other views. For example, consider a case where the first view consists of high-resolution images, while the second view consists of low-resolution images. In the field of super resolution field, it is common to use high-resolution images (ideal view) to supplement information and guide the learning of low-resolution images (other view), e.g., 2019-ICCV-Guided Super-Resolution as Pixel-to-Pixel Transformation and 2021-CVPR-Robust Reference-based Super-Resolution via C2-Matching. Similarly, we select one view as the reference ("ideal") view to supplement and correct the other views.

3. Previous works have regarded data partially align as noisy. During the model's testing phase, they use an alignment strategy to align the $v-1$ views to the first view, thereby fusing multi-view feature for clustering, e.g., CANDY (line 53 of https://github.com/XLearning-SCU/2024-NeurIPS-CANDY/blob/main/model.py) and RMCNC (line 236 of https://github.com/sunyuan-cs/2024-TKDE-RMCNC/blob/main/RMCNC_main/sure_inference.py). This alignment operation implies that these papers consider the first view as an ideal view. Therefore, although the scenario settings may differ, to maintain generality, we also treat the first view as an ideal view.

Explanation for baselines: Thanks. Previous studies consider data partially align as noisy. In our work, we extend the definition of noise and explore a more common noisy scenario, where noise exists within individual views. Recently, MVCAN is the only work that explores the issue of noisy views, leaving no other methods available for direct comparison. MVCAN incorporated comparisons with numerous contrastive learning-based methods. Following this setup of MVCAN, we evaluated the performance of various algorithms under our proposed noisy setting. To further validate the effectiveness of our approach, we included the latest multi-view clustering methods from 2024 in Tables 1 and 2 of our submitted version. Moreover, our selection of a substantial number of contrastive learning-based methods is that contrastive learning could enhance both the model's robustness and discriminative capability. Therefore, in the absence of directly comparable methods, we select contrastive learning-based methods to demonstrate the effectiveness of our method.

Explanation for balanced clusters: Thanks. Cluster balance is a widely adopted default assumption in clustering problems, and we follow this common assumption as well. Additionally, to further verify the cluster balance of samples, we conduct statistical analyses on the datasets used in AIRMVC. The results indicate that the sample classes in the utilized datasets are nearly balanced. Due to space limitations, detailed results can be found in Tab.2 in https://anonymous.4open.science/r/Res-11B2.

Typos & Presentation: Thanks. Following your suggestion, we will correct the typos and further improve the presentation.

Experiments of time and space cost: Thanks. Following your suggestion, we conducted time and space complexity experiments on the six used datasets with 10% noisy ratio. Specifically, we measure the training time per epoch for all baselines using seconds as the evaluation metric. The space cost experiments are conducted on an NVIDIA A6000 GPU, measured in gigabytes (GB). The results are presented in Tab.1 and Tab.2. From these results, we observe that the time and space costs of AIRMVC remain within an acceptable range. In summary, AIRMVC demonstrates promising clustering performance while maintaining a reasonable computational cost.

Tab.1 Time cost for AIRMVC.

Tab.2 Space cost for AIRMVC.

Explanation for symbol in Fig.2: Thanks. In Fig. 2(a), (b), and (c), the dark blue color represents noisy data. Following your suggestion, we will provide a more detailed explanation in the final version.

Typos & Format: Thanks. Following your suggestion, we will revise RQ3 and Eq.8 in the final version and review similar issues to enhance the overall presentation.

Details for visualization experiments: Thanks. We visualized the latent space features extracted by the encoder from the UCI-Digit dataset using the t-SNE algorithm. The visualization was performed every 20 epochs for the first 200 epochs. The experiments were conducted on an NVIDIA A6000 platform. We will provide additional descriptions in the future.