An Effective and Secure Federated Multi-View Clustering Method with Information-Theoretic Perspective

Thank you for your valuable comments and suggestions.

Q1: In order to analyze their impact to the final performance, could the author provide the hyperparameter experiment about $\mathcal{L}^m$ and $\mathcal{L}_a^m$ of Equation 12? Even if the author do not tune hyperparameter in their experiments.

A1: Thank you for your suggestion. We redefine Equation 12 as $\mathcal{L}^m_{inc}=\mathcal{L}^m+\beta\mathcal{L}_{a}^m$ and analyze the hyperparameter $\beta$. The table below shows the sensitivity of this parameter by varying $\beta$ the range on $[10^{-2},10^{2}]$ the BDGP dataset in incomplete scenarios. The results show that the model achieves the best performance at $\beta=1$, reflecting the fact that the two parts of Equation 12, $\mathcal{L}^m$ and $\mathcal{L}_a^m$, are similarly weighted and are both important for model optimization, so the hyperparameter is not tuned in our experiments.

Q2: How does the proposed method address potential threats from malicious clients, as it currently only demonstrates effectiveness against reconstruction attacks on the server side?

A2: Thank you for your careful reminder. Our proposed method primarily addresses model attack threats caused by semi-honest participants. Although we did not explicitly delve into defenses against data attacks initiated by dishonest participants such as malicious clients, our method remains effective in such cases. Specifically, if there exists malicious clients with false or manipulated data, their local clustering structures are likely to significantly deviate from that of other clients. By comparing the local cluster assignments or centroids uploaded by each client, the server can set a malicious threshold to flag outliers. If a client's data exceed this threshold, its shared information can be disregarded, and the client can be marked as malicious, effectively mitigating such attacks.

Q3: Could the authors integrate the discussion on extending the method to cross-device scenarios (currently in the appendix) into the main text and provide more in-depth analysis?

A3: Thank you for your suggestion. We will include experimental results and a discussion on extending to cross-device scenarios in Section 5.3. We believe that enhancing the scalability of the method to adapt to various scenarios is an essential area of exploration. In the new version, we will present additional experimental results on more multi-view datasets and provide further analysis of these results. Furthermore, we have envisioned a solution for the scenario where the sample size per client becomes insufficient for effective training as the number of clients increases. To maintain good performance, we will consider strategies such as continuing training based on existing models or sharing partial local model information to alleviate the issue of insufficient samples per client, enabling collaborative training across multiple clients.

Q4: Section 4, titled "Discussion and Analysis," seems to lack discussion on key findings—could the authors elaborate further to strengthen this section?

A4: Thank you for your feedback. In the new version, we plan to add two parts discussing model generalization and privacy protection. Regarding model generalization, we will focus on the scalability of the proposed method, including extensions to incomplete and cross-device scenarios. These scenarios primarily aim to limit shared information, only relying on the model’s generalization ability. For example, for incomplete scenarios, each client uploads clustering-related features for overlapping samples to the server, while non-overlapping samples are clustered locally. For cross-device scenarios, each client uploads clustering-related features extracted from local data, with the server aligning overlapping samples. Our method leverages generalization to perform well with a few extensions. Regarding privacy protection, we will discuss the privacy-preserving scenarios addressed by our method and potential strategies for further enhancing privacy. Our method is primarily designed for environments where all participating parties are semi-honest, meaning they faithfully execute the training protocol but may attempt privacy attacks. Currently, our feature splitting strategy is sufficient to defend against common model inversion attacks in such settings. For further privacy enhancement, we could integrate commonly used privacy-preserving techniques in federated learning, such as differential privacy or homomorphic encryption, to offer additional privacy protection.

We thank the reviewer for valuable comments and suggestions that have greatly improved our paper.

Q1: While the paper focuses on information-theoretic privacy preservation, it does not provide a detailed comparison with other commonly used privacy-preserving techniques in federated learning.

A1: Thank you for your valuable feedback. Our method can be integrated with commonly used privacy-preserving techniques in federated learning to further enhance privacy protection. Specifically, for clustering-related features shared across clients, our feature splitting strategy already prevents common attacks, such as model inversion attacks, preventing attackers from reconstructing the original data. However, for stronger privacy guarantees, additional privacy-preserving techniques can be added. For instance, as referenced in our response to Reviewer qNsU Q3, we report the impact of differential privacy on our method’s performance under different privacy budgets. Additionally, our method aims to balance privacy protection and clustering performance. Excessive privacy constraints inevitably lead to performance degradation, as confirmed by our experimental results. Similarly, homomorphic encryption could be integrated with our method to further enhance privacy protection, though at the cost of increased computational overhead in both training and inference.

Q2: The authors do not provide a clear definition or description of the privacy under the federated multi-view clustering scenario.

A2: Thank you for your feedback. We describe the privacy protection scenario of ESFMC in Lines 416–418: we assess whether all semi-honest participants can reconstruct the original data through certain attack methods based on shared information. However, this may not be explicitly stated. To clarify, we will restate ESFMC’s privacy protection scenario in the problem statement: we assume that all participating parties are semi-honest and do not collude. An attacker follows the training protocol but may attempt privacy attacks to infer private data from other parties.

Q3: What is the advantage of the collaborative alignment strategy over the cross-view alignment strategy based on adaptively calculate alignment matrices in FCUIF (Ren et al., 2024)?

A3: First, in terms of performance, our proposed method demonstrates superior effectiveness in handling incomplete scenarios compared to FCUIF on the same datasets (BDGP and Scene), achieving better results. Second, from a methodological perspective, FCUIF leverages sample commonality and view versatility, enabling the server to adaptively compute alignment matrices for cross-view alignment. In contrast, our collaborative alignment strategy with an information-theoretic perspective, aligns features by maximizing mutual information. Compared to FCUIF, our method serves as a generalizable interface module that can be integrated into other methods. It offers greater adaptability and scalability while also achieving improved performance.

Q4: To optimize $\omega^m_{t,k}$, why traditional SGD is not suitable?

A4: Traditional SGD seeks a single optimal point estimate of the parameters by minimizing the loss function. In contrast, our method focuses on the posterior distribution of the parameters rather than a single estimate. To achieve this, we employ SGLD, which integrates SGD with Langevin dynamics by introducing noise into the gradient updates. This added noise facilitates sampling from the posterior distribution rather than converging to a single mode.

Q5: For privacy preservation, if the work aims to minimize the information to be shared, a straightforward way is to minimize $I(X^m;Z_c^m)$, but it seems that the method uses $I(Z_x^m, Z_c^m)$ instead.

A5: This is an interesting perspective. Minimizing $I(X^m;Z_c^m)$ is indeed a more direct approach to reducing the amount of sensitive information from $X^m$ in the shared information $Z_c^m$. However, we choose to minimize $I(Z_x^m, Z_c^m)$ to emphasize the feature splitting strategy better. We aim to ensure that the extracted features, $Z_x^m$ and $Z_c^m$, serve distinct purposes with minimal redundancy, thereby achieving high-quality feature splitting. We believe that both optimization strategies serve a similar goal, differing primarily in their formulation and optimization approach.

We sincerely appreciate your constructive comments and suggestions.

Q1: Have you considered evaluating ESFMC on real-world federated datasets,to better assess its applicability in practical scenarios?

A1: Thank you for your suggestion. We have considered evaluating ESFMC on real-world datasets and have obtained some preliminary results. The Organ{A,C,S}MNIST [1] dataset is derived from the liver tumor segmentation benchmark (LiTS), which consists of 3D computed tomography (CT) scans. We generate 2D images by extracting center slices from the 3D bounding box in the axial, coronal, and sagittal planes, corresponding to three visual views, resulting in a total of 13,000 samples. Below are our experimental results:

[1]Yang J, Shi R, Wei D, et al. MedMNIST v2-A large-scale lightweight benchmark for 2D and 3D biomedical image classification. Scientific Data, 2023, 10(1): 41.

Q2: More extensive empirical experiments—such as testing ESFMC against more sophisticated adversarial attacks (e.g., gradient inversion, membership inference attacks).

A2: Thank you for your valuable comment. Our primary focus is on privacy verification against model inversion attacks, which are among the most common threats targeting shared intermediate results. These attacks leverage model outputs to reconstruct the original input data and are a type of sophisticated adversarial attack. They are also one of the most prevalent and effective attacks in federated learning settings that share intermediate results (such as the features in our paper).

Regarding the other attacks you mentioned, they may not be applicable to our federated learning scenarios. For instance, gradient inversion attacks reconstruct original data by exploiting gradient information, which is commonly observed in federated learning settings that share model parameters or gradients. However, in our method, all client model training parameters remain strictly local, making such an attack infeasible. Similarly, membership inference attacks aim to determine whether a specific sample was part of the training dataset, thereby probing data privacy. However, since our training data are entirely unlabeled and does not include explicit sample membership information, attackers would gain no meaningful insights merely by inferring whether a sample participated in the training.

Additionally, we wish you to refer to our response to Reviewer pjmK Q2, where we elaborate on how our proposed method addresses potential threats from malicious clients, further strengthening its ability to safeguard sensitive information across different scenarios. Lastly, it is important to highlight that our method specifically tackles the trade-off between privacy preservation and performance improvement. Unlike existing works, which typically focus on one aspect at the cost of the other, our approach strives to maintain strong performance while minimizing privacy leakage as much as possible.

Q3: Can you provide additional quantitative results on privacy guarantees, such as empirical differential privacy noise impact analysis?

A3: Yes, we have included additional quantitative results on privacy guarantees. Specifically, we incorporate differential privacy by adding noise to the clustering-related features uploaded from clients to the server. The table below presents the clustering performance of ESFMC under different privacy bounds $\varepsilon$ on Caltech dataset. We observe that ESFMC achieves both high performance and privacy at $\varepsilon=50$. However, as the level of noise increases at $\varepsilon=25$, the performance of ESFMC unavoidably degrades.

We sincerely thank the reviewer for these valuable comments.

Q1: The paper lacks results on large-scale datasets.

A1: We conduct further experiments on the large-scale YoutubeVideo dataset, which contains 101,499 samples across 31 classes, where each sample has three views of cuboids histogram, HOG, and vision misc. The clustering results of ESFMC and several comparison methods in incomplete scenarios are shown below:

We select the YoutubeVideo dataset, which is 20 times larger than the Scene dataset with 4,485 samples. The results demonstrate that our method adapts well to large-scale datasets and outperforms other methods in terms of performance, ensuring the proposed method's robustness and broader applicability.

Q2: In the incomplete multi-view setting, more scenarios of data heterogeneity, such as quantity skew, should also be explored.

A2: Thank you for your suggestion. To explore the scenario of data heterogeneity, such as quantity skew, we introduce Dirichlet distribution when constructing incomplete datasets. A smaller Dirichlet parameter $\alpha$ leads to more heterogeneous splits, resulting in highly imbalanced sample sizes among clients. The table below presents three levels of heterogeneity by setting $\alpha$ to $10^{-2}$ (high), $10^{0}$(moderate), and $10^{2}$ (none) on BDGP dataset in incomplete scenarios. The results demonstrate that ESFMC maintains strong performance even under high heterogeneity, with only a slight performance drop.

Q3: How do the authors evaluate the correctness of their feature-splitting strategy?

A3: This is indeed an issue worthy of attention. Our feature splitting strategy is guided by an intuitive training loss function. Specifically, $\mathcal{L}^m_{1}$ ensures that sample-related features accurately reconstruct the original data, $\mathcal{L}^m_{2}$ encourages clustering-related features to capture meaningful clustering structures, and $\mathcal{L}^m_{3}$ minimizes redundancy between different feature types, promoting high-quality feature splitting. In the ablation studies (Table 3), removing $\mathcal{L}^m_{1}$ and $\mathcal{L}^m_{3}$ in the variants results in performance degradation, demonstrating that the feature splitting strategy effectively splits different features, thereby ensuring the effectiveness of our method. Furthermore, in Table 5, we analyze the impact of sharing different types of features on clustering performance. The results indicate that the clustering-related features extracted through our feature splitting strategy are more effective and accurate in capturing clustering structures, further validating the strategy’s accuracy and effectiveness. Overall, these ablation studies confirm that our feature splitting strategy successfully separates sample-related and clustering-related features, significantly enhancing clustering performance.