Evaluate then Cooperate: Shapley-based View Cooperation Enhancement for Multi-view Clustering

1. Illustration in Figure 2:

Thanks. Improvements have made to the View Cooperation Enhancing Module in Figure 2 in the Figure 1 of global reponse, presenting the details of gradient modulation within this module.

2. Notification table:

Thanks for your suggestions. A notification table will be given in the final version.

3. Key to improving model performance:

Thanks. Previous DMVC methods did not adequately consider the aspects of view cooperation, especially the joint methods, which may lead to the collapse of fusion results onto one or a few views. Building upon the fundamental assumption of multi-view learning that each view contains complementary information beneficial for downstream tasks, fully leveraging the role of each perspective and enhancing cooperation leads to better model performance.

1. the use of SCE module on alignment-based methods:

Thanks. The SCE method consists of two modules: the View Contribution Evaluation Module and the View Cooperation Enhancing Module. For alignment-based methods, the View Contribution Evaluation Module obtains the contributions of views, where these contributions are remarkably close in the alignment-based frameworks, agreeing with the results deduced from our theory.

Building upon the consistent view contributions, the limited enhancements brought by the View Cooperation Enhancing Module can be elucidated. Our experimentation and analysis of alignment-based methods with/without SCE in the paper are not aimed at showcasing the enhancement brought by SCE to the model, but rather at validating the integrity of our theory and the soundness of the View Contribution Evaluation Module.

2. Is the alignment-based method superior to the joint method?

Thanks. Given a DMVC framework, the SCE module can evaluate the cooperation among views and make optimizations based on the assessment. If the cooperation among views is insufficient, it indicates potential for improvement upon the original clustering performance, while it does not imply that the original clustering results are poor. For example, on the Caltech101-7 dataset, while DMJC may not match ProImp in view cooperation, it exhibits superior clustering performance. Moreover, the inconsistent view cooperation of DMJC suggests that using the SCE module can yield remarkable enhancements to the model. Our work, from the perspective of view contributions, highlights potential limitations of joint methods, and does not assert that alignment-based methods are superior than joint methods in clustering performance.

3. Algorithmic complexity of SCE module:

Thanks. SCE module involves computing the fusion distribution for any combination of views. With V views, the complexity of SCE amounts to $O(2^V)$. While the $O(2^V)$ complexity is manageable for a small number of views, it can become cumbersome with a larger number of views. To mitigate the computational burden of calculating Shapley values for numerous views, approximate algorithms can be employed. For instance, the TreeSHAP [1][2] algorithm can compute Shapley values in polynomial time, offering a more efficient approach.

4. Representation of symbols in the formula:

Thanks. The $L$ in Equation 15 is an abbreviation for $L(\theta^{(v)}_t)$ in Equation 16. The representation of the formulas will be unified in the final version.

Reference

[1] Yang J. Fast treeshap: Accelerating shap value computation for trees[J]. arxiv preprint arxiv:2109.09847, 2021.

[2] Muschalik M, Fumagalli F, Hammer B, et al. Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(13): 14388-14396.

1. Quantification of the equilibrium level of view contributions:

Thanks. Due to the normalized characteristic of view contributions, the cooperation level among views with/without SCE can be compared by calculating the variance of view contributions, denoted as $D(\phi)$. A smaller variance indicates a more consistent contribution among views, reflecting improved cooperation between views. The table below illustrates the values of $D(\phi)$ for DMJC with/without SCE. It is evident that using SCE could result in a lower $D(\phi)$ for the views, indicating a better cooperation among them.

2. Physical meaning of view contributions:

Thanks. In unsupervised multi-view scenario, the physical meaning of view contributions lies in their influence on fusion. The greater the view's contribution, the more significant its influence on the fusion progress.

3. Relationship between view contribution and view performance:

Thanks. There is no link between the contribution of a view and its performance. Based on the fundamental assumption of multi-view learning that each view contains complementary information beneficial for downstream tasks, examining the performance of a single view independently is deemed meaningless. Furthermore, it is infeasible to evaluate the quality of a single view under unsupervised conditions. Hence, the aim is to enhance the cooperation among views by ensuring a more balanced view contribution.

1. Different characteristics of joint method (Figure 3(a)) and alignment-based method(Table 2):

Thanks. Figure 3(a) illustrates the change of view contributions of DMJC (a joint method) with/without SCE. In the joint methods, views' representations are optimized in their respective spaces, leading to uneven contributions in the fusion process. Even with the utilization of the SCE module, it can only alleviate the imbalance in view contributions without guaranteeing complete consistency. This is an inherent characteristic of the joint method framework.

On the other hand, the two alignment-based methods in Table 2 inherently bring views' representations closer, leading to more consistent contributions between views. With the application of the SCE module, it is possible to achieve essentially consistent view contributions. These experimental results validate the theoretical framework proposed in our study.

2. Criteria of classifying DMVC:

Thanks. Our classification of DMVC methods is inspired by [1], which categorizes multi-view representation learning into (1) joint methods, (2) alignment methods, and (3) shared and specific methods, based on whether the representations of views are brought closer in the same space. In shared and specific methods, shared representations are brought closer to each other in the same space, while specific representations are individually optimized in different spaces, which can be seen as a combination of (1) and (2). Similar classification methods are also found in [2], [3], [4]. Drawing on this classification approach, our work divides DMVC into joint methods, alignment-based methods, and other methods.

3. Similar frameworks to DMJC:

Thanks. The methods of DMJC have found numerous applications in MVC research in recent years. Approaches such as SURER [5], SGDMC [6], and DFP-GNN [7] have also employed similar strategies, namely optimizing view representations in respective spaces and using the sharpening of fused distributions as self-supervised signals to guide training.

Reference

[1] Jia X, et al. Human collective intelligence inspired multi-view representation learning—Enabling view communication by simulating human communication mechanism[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(6): 7412-7429.

[2] Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: A survey and taxonomy[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 41(2): 423-443.

[3] Li Y, Yang M, Zhang Z. A survey of multi-view representation learning[J]. IEEE transactions on knowledge and data engineering, 2018, 31(10): 1863-1883.

[4] Jia X, et al. Semi-supervised multi-view deep discriminant representation learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 43(7): 2496-2509.

[5] Wang J, Feng S, Lyu G, et al. SURER: Structure-Adaptive Unified Graph Neural Network for Multi-View Clustering[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(14): 15520-15527.

[6] Huang Z, Ren Y, Pu X, et al. Self-supervised graph attention networks for deep weighted multi-view clustering[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(7): 7936-7943.

[7] Xiao S, Du S, Chen Z, et al. Dual fusion-propagation graph neural network for multi-view clustering[J]. IEEE Transactions on Multimedia, 2023, 25: 9203-9215.

1. The relationship between $\phi$ and $w$:

Thanks. Evaluating the view contribution using $\phi$ calculated by shapley value, instead of relying solely on pre-set weights $w$, stems from a systemic perspective on the process of multi-view clustering. In multi-view of representation learning and fusion, conventional static weights ($w$) overlook the intrinsic interactions among views. Contrarily, Shapley values’ distinctive marginal analysis capability allows for precise quantification of each view’s average incremental contribution to the system’s overall performance. Therefore, $\phi$ reveals not only the fundamental worth of each view, but also showcasing the additional value they generate through dynamic interactions.

2. Pushing away contributions:

Thanks. Pushing away contributions among different views can be seen as the antithesis of our SCE module. In this context, the fusion of views collapses onto one or a few views, failing to leverage the complementary information of multi-view information.

3. Counterexamples:

Thanks. There are situations where the contributions of views are brought closer together without enhancing the clustering performance. The fundamental assumption of DMVC is: Each view contains complementary information beneficial for downstream tasks, better view cooperation leads to better model performance. However, if a dataset obeys this assumption that a certain view contains a significant amount of noise and erroneous information, detrimental to the clustering task, increasing the contribution of that view will result in an overall decrease in clustering performance.

Thanks for your responses, my questions have been resolved. Additionally, I checked out the feedback of other reviewers. I think the motivation of this paper commendable. I decide to elevate the rating of the paper to weak accept.