Federated Generative Learning with Foundation Models

Q4: The framework hasn't been benchmarked against other federated learning methods that employ generative models.

A4: Unfortunately, we were unable to find any existing methods in the literature that directly address our specific setting, making it difficult to perform a fair comparison. In light of this, and following the suggestions of other reviewers, we conducted experiments using various types of generative models to demonstrate the applicability of our proposed method. To investigate the impact of various generative models on the results, we followed the setting in [1]. Our experiments primarily focus on three prevalent conditional diffusion models: DiT[2], GLIDE[3], and Stable Diffusion. We use these off-the-shelf models to generate synthetic images. Specifically, for GLIDE and Stable Diffusion, the prompt was configured as "a photo of {label name}, real-world images, high resolution." For DiT, the input comprised the label ID corresponding to the ImageNet1k dataset. The images synthesized by DiT and GLIDE are of dimensions 256x256, whereas those produced by Stable Diffusion are of dimensions 512x512. As shown in the following table, even when we vary the foundation models used in our method, FGL consistently outperforms FedAvg by a significant margin. This observation serves as evidence for the generality of our approach. We have added these results in the appendix.

[1] Li, Zheng, et al. "Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?." arXiv preprint arXiv:2305.12954 (2023).

[2] Nichol, Alex, et al. "Glide: Towards photorealistic image generation and editing with text-guided diffusion models." arXiv preprint arXiv:2112.10741 (2021).

[3]Peebles, William, and Saining Xie. "Scalable diffusion models with transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Thank you for your constructive comments. We hope the following clarifications can address your concerns.

Q1: The evaluation of the Federated Generative Learning (FGL) framework is limited to simpler domains like ImageNet and doesn't extend to other areas, casting doubt on whether prompts can encapsulate complexity.

A1: To further validate the effectiveness of our method, we conducted experiments on several fine-grained image classification datasets, namely CUB-200, Stanford Cars, and also the satellite image dataset EuroSAT. CUB-200 is a challenging dataset consisting of 200 bird species, while Stanford Cars contains 16,185 images belonging to 196 classes of cars. As for EuroSAT, the official dataset did not provide predefined training and testing splits, so we performed a split in an 8:2 ratio. The size of fine-grained recognition datasets is typically smaller compared to general image classification datasets. In previous work, a common practice is to utilize a pretrained model that has been trained on the ImageNet dataset. In this study, we present two approaches: training the model from scratch and loading a pretrained ResNet34 model. As shown in the table, our method achieves excellent performance even in these challenging domains. This can be attributed to the fact that regardless of the magnitude of domain differences, pretraining a well-performing model on our synthetic data is beneficial for the downstream federated tasks. We have added these results in the appendix.

Q2: While FGL aids in data generation for non-IID data, achieving congruence with a global distribution is yet to be addressed.

A2: Thank you for your valuable feedback. We acknowledge that FGL is effective in generating data for non-IID scenarios, aligning it with a global distribution (e.g., IID settings) also works in our experiments (see Table 1 for IID results).

Q3: Security risks of prompts require more analysis. Could prompts be reverse-engineered to obtain private data?

A3: During the communication phase, traditional Federated Learning (FL) methods typically transmit model parameters or gradients. However, these parameters can be vulnerable to adversarial attacks and model inversion attacks if intercepted by an adversary. To enhance security, some FL methods utilize prompts for communication. However, the potential for attackers to reconstruct private data using prompts has received limited research attention, both in black-box and white-box scenarios. Recent work[1,2] has identified risks associated with the reconstruction of pretrained data in Diffusion models. Nevertheless, there is currently no available method that can solely reconstruct previously unseen private data in Diffusion models based solely on prompts. This presents an interesting and promising research direction for future investigations. Consequently, considering the lack of research in this area, our method can be regarded as relatively safe and privacy-preserving.

[1]Shen, Xinyue, et al. "Prompt Stealing Attacks Against Text-to-Image Generation Models." arXiv preprint arXiv:2302.09923 (2023).

[2]Carlini, Nicolas, et al. "Extracting training data from diffusion models." 32nd USENIX Security Symposium (USENIX Security 23). 2023.

Dear Reviewer,

Thank you once again for your valuable comments. As the discussion stage is coming to a close in 2 days, we kindly request your feedback on whether our response adequately addresses your concerns. We would greatly appreciate any additional feedback you may have.

Thank you in advance!

Kind regards,

Authors

Dear Reviewer zu8q,

As the discussion period is closing, we sincerely look forward to your feedback. The authors deeply appreciate your valuable time and efforts spent reviewing this paper and helping us improve it.

Please also let us know if there are further questions or comments about this paper. We strive to improve the paper consistently, and it is our pleasure to have your feedback!

Best regards,

Authors

Q4: The comparison to a single baseline, FedAvg, falls short; including comparisons to advanced Federated Learning frameworks could better highlight the proposed method's effectiveness.

A4: We have compared the two popular FL methods, Moon[1] and Fedopt[2]. We conducted experiments on the ImageNette and ImageNet100 datasets, considering a scenario with 50 clients under non-IID settings (beta=0.5). To the best of our knowledge, there is currently no federated learning method that surpasses centralized training. However, our proposed method even outperforms centralized trained models in many scenarios (see Table 1 in main text). Therefore, as shown in this table, our method still outperforms other federated learning approaches.

[1]Li, Qinbin, Bingsheng He, and Dawn Song. "Model-contrastive federated learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.

[2]Reddi, Sashank J., et al. "Adaptive Federated Optimization." International Conference on Learning Representations. 2020.

Q5: Table 2 shows the proposed method outperforming centralized learning significantly; a thorough explanation of this phenomenon is warranted.

A5: This is because our method synthesizes a balanced dataset using all collected prompts during the first round of communication. We then pretrain a "well-initialized" model on this dataset. Once we have this well-initialized model, several rounds of communication can quickly bring the model to a good performance. In the first table, we present the results of directly loading a pretrained model on ImageNet. It can be observed that directly loading a pretrained model on ImageNet reduces the gap between our method and FedAvg. This is because the pretrained model provides a good starting point. However, training a pretrained model on ImageNet requires a significant computational cost on a dataset of 1.3M samples. In contrast, our method only requires training on a small amount of synthesized data to provide a well-initialized model, hence achieving better performance than models trained in a centralized manner.

Thank you for your valuable time in reviewing our paper. Below are responses to your concerns. Please let us know if you require any further information, or if anything is unclear.

Q1: The training framework is predominantly tailored for image datasets, limiting its applicability.

A1: Our approach is based on existing generative models that are widely used in various domains, such as Computer Vision with Stable Diffusion and Natural Language Processing with GPTs. This means that our framework can easily be applied to other domains, including NLP. However, due to time constraints, we were unable to conduct additional experiments on NLP tasks. We believe that further research in this area would be valuable and should be pursued in the future.

Q2: The method heavily depends on the congruence between the captioning and generative models, making it challenging to ensure the proxy dataset's distribution aligns with the private data.

A2: To further validate the effectiveness of our method, we conducted experiments on several fine-grained image classification datasets, namely CUB-200, Stanford Cars, and also the satellite image dataset EuroSAT. CUB-200 is a challenging dataset consisting of 200 bird species, while Stanford Cars contains 16,185 images belonging to 196 classes of cars. As for EuroSAT, the official dataset did not provide predefined training and testing splits, so we performed a split in an 8:2 ratio. The size of fine-grained recognition datasets is typically smaller compared to general image classification datasets. In previous work, a common practice is to utilize a pretrained model that has been trained on the ImageNet dataset. In this study, we present two approaches: training the model from scratch and loading a pretrained ResNet34 model. As shown in the table, our method achieves excellent performance even in these challenging domains. This can be attributed to the fact that regardless of the magnitude of domain differences, pretraining a well-performing model on our synthetic data is beneficial for the downstream federated tasks.

Q3: The experimental setup, with only five clients, may not adequately represent real-world scenarios; expanding the evaluation to include 50 or 100 clients could provide more insightful results.

A3:Thanks for your suggestion. We extended our analysis to include the results obtained from the ImageNette dataset with 50 and 100 clients. As depicted in the table, our method continues to exhibit superior performance compared to FedAvg across all scenarios. Additionally, the improvements achieved by our method remain significant. See more details in the Appendix.

Dear Reviewer,

Thank you once again for your valuable comments. As the discussion stage is coming to a close in 2 days, we kindly request your feedback on whether our response adequately addresses your concerns. We would greatly appreciate any additional feedback you may have.

Thank you in advance!

Kind regards,

Authors

Q4: What exactly is client privacy in this case? Can the client data be still considered “private” if you could already generate them with public foundation models.

A4: client privacy: In this paper, similar to differential privacy, we primarily focus on individual privacy, as it is more challenging for attackers. For instance, Attack A perfectly targets a known subset of 0.1% of users in a client, but succeeds with a random 50% chance on the rest. Attack B succeeds with a 50.05% probability on any given user in a client. On average, these two attacks have the same attack success rate. However, the second attack is practically useless, while the first attack is much more powerful in the real-world. This is precisely what LiRA[1] emphasizes, as it evaluates the privacy attack by computing their true-positive rate at very low (e.g., ≤ 0.1%) false-positive rates (as illustrated in our experimental results in Figure 8), demonstrating that our method can better defend against privacy attacks.

[1] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.

Q5: In many cases, the descriptions of the images can already be leaking privacy.

A5:This could be the difference between individual privacy and group privacy. The majority of the current papers on data protection focuses on the individual ‘user,’ or ‘data subject,’ who’s right to privacy will grow exponentially with the enforcement of the GDPR (General Data Protection Regulation). However, group privacy is not mentioned in the GDPR , which is not well-defined. Also, if the server knows some of the user data distribution means potential privacy risks, our proposed method does not introduce additional risk in this regard. This is because in traditional gradient/parameter-based methods, using model inversion, the server can still infer this information[1,2]. But for individual privacy, this information won't increase the leakage of membership in private data.

[1] Geiping, Jonas, et al. "Inverting gradients-how easy is it to break privacy in federated learning?." Advances in Neural Information Processing Systems 33 (2020): 16937-16947.

[2] Hatamizadeh, Ali, et al. "Do gradient inversion attacks make federated learning unsafe?." IEEE Transactions on Medical Imaging (2023).

Q6:Why exactly does the proposed method provide robustness to data heterogeneity? Heterogeneity can still surface in the (instance-level) client prompts and subsequently the generated images.

A6: For one-shot Federated Learning (FL), regardless of the extreme data distributions among different clients, the server can always collect prompts corresponding to all the data, thus obtaining a balanced synthetic dataset on the server. Therefore, compared to FedAvg, our method is not sensitive to data heterogeneity in the first round of communication. In the subsequent model updates, the clients are still affected by non-IID data. However, due to the well-trained initial model obtained in the first round of communication, only a few rounds of communication are needed for local updates, making it more robust to data heterogeneity. As shown in Table 1 in the main text, our method exhibits significantly smaller gaps compared to FedAvg under different non-IID scenarios.

Q7: Minor comment: consider using different citation commands \citet , \cite

A7: Thanks for pointing this out. we will check it in the updated version.

We thank Reviewer wnsW for the valuable feedback and insightful comments. Here, we answer your questions and provide more experimental evidence.

Q1: The datasets used by the experiments are all natural image datasets (ImageNette, ImageFruit, etc.), which can be well-represented in the pre-training dataset of foundation generative models. I would appreciate results on non-natural image datasets.

A1: Although the pretraining dataset and the private dataset may exhibit some domain similarities (e.g., both may contain common real-world scenes), the tasks of lmageSquawk (fine-grained bird classification) and QuickDraw (non-realistic domain) in DomainNet that we demonstrate in our experiments are inherently challenging. Training a model solely on synthetic data generated by foundation models to achieve high accuracy on the ImageNet or DomainNet test sets is a non-trivial task. To further validate the effectiveness of our method, we conducted experiments on several fine-grained image classification datasets, including CUB, Cars, and the satellite image dataset EuroSAT. As the official EuroSAT dataset did not provide predefined training and testing splits, we performed a split in an 8:2 ratio. The size of fine-grained recognition datasets is typically smaller compared to general image classification datasets. In previous work, a common practice is to utilize a pretrained model that has been trained on the ImageNet dataset. In this study, we present two approaches: training the model from scratch and loading a pretrained ResNet34 model. As shown in the table, our method achieves excellent performance even in these challenging domains. Additionally, in the cross-silo federated learning scenario, when clients have strong computational capabilities, one can simply finetune the foundation models on these domains, achieving better performance than normal federated learning methods. We have added these results in the appendix.

Q2: the “class-level prompts” may not be needed at all since the server can just generate images by itself.

A2: Yes, if the server-side has knowledge of the specific labels for the classification task, it can generate them directly. However, class-level is just a simple case. We propose the instance-level approach to address more complex domains, where client-side customized prompt generation is more advantageous in improving the performance of the overall model.

Q3: More broadly, the threat model of the paper may need to be defined more clearly.

A3: threat model: In traditional federated learning schemes that transmit model parameters/gradients, attackers can launch various attacks once they obtain the parameters/gradients, such as membership inference attacks, adversarial example attacks, and model inversion. In contrast, our approach significantly reduces potential security and privacy risks because users only transmit prompts in the first round of communication. To the best of our knowledge, there is no research indicating that using prompts alone can perfectly reconstruct private data. Therefore, our approach is more secure and privacy-preserving compared to FedAvg.

Dear Reviewer,

Thank you once again for your valuable comments. As the discussion stage is coming to a close in 2 days, we kindly request your feedback on whether our response adequately addresses your concerns. We would greatly appreciate any additional feedback you may have.

Thank you in advance!

Kind regards,

Authors

I appreciate the authors for providing a rebuttal.

• A1: I appreciate the authors for putting efforts into the new results. I also appreciate pointing to the results on QuickDraw. However, my concern is not fully addressed since the datasets “CUB-200” (natural images of birds) and “Cars” (natural images of cars) are very much still in-distribution for the pre-trained generative vision models.
• A2: The authors responded to my question by pointing to the use of instance-level prompts, but this didn’t quite address my concern that the significance of the class-level prompts is a bit overclaimed. Considering the default implementation of your experiments uses class-level prompts (page 6), I would suggest clearly spelling out the assumptions and weaknesses of class-level prompts in the updated paper.
• A4/A5:
  • (For clarity, the following discussions apply to “instance-level” prompts)
  • By explaining the LiRA paper on membership inference in A4, the authors imply that the paper cares about instance-level privacy — i.e. image-level privacy, where an attacker cannot confidently tell whether one image is or isn’t used for training.
  • I’m definitely okay with the privacy granularity in this case; what I’m uncertain about (with Q5) is whether all the information contain within a single example (i.e. image-label pair) is protected.
  • A5 does not quite address my question. I do not agree that this is the difference between “group privacy” vs “individual privacy”; rather it is that the instance-level prompts have provided side channels into learning about the information of a single image.
  • Consider running local, image-level DP-SGD on a client when participating in a vanilla FedAvg task. All the information corresponding to a single example (pixel values and labels) are protected behind the “privacy barrier” since privatized gradients are applied to the model. In contrast, instance-level prompts would leak information about the pixel values, and thus do not really satisfy instance-level privacy in the sense of “attacker not being able to tell whether an image is used for training”. I do acknowledge however that there is value in providing empirical privacy of the pixel values.
• A6: Thanks for the clarification that the server can select/curate prompts to essentially manually mitigate the data heterogeneity. I would suggest highlighting this in the updated version.

Overall, the technique proposed in the paper is interesting, though I feel the assumptions on client data distributions and privacy claims are too strong. Having read through other reviewers’ comments, I’m keeping my rating at 5.

Q3.Evaluation on ImageNet or DomainNet implicitly uses the assumption that local data have a similar or subset domain to the pretraining dataset of foundation models, which are publically accessible or have no privacy issue.

A3: Thanks for pointing this out. Here, we would like to address this from three aspects:

• Although the pretraining dataset and private dataset may have some domain similarities (e.g., both may contain common real-world scenes), the tasks of lmageSquawk (fine-grained bird classification) and QuickDraw (non-realistic domain) in DomainNet that we show in our experiments are challenging. It is a non-trivial task to train a model only using synthetic data generated by foundation models to achieve good accuracy on the ImageNet or DomainNet test sets.

• Furthermore, even when there are some domain similarities between the pretraining dataset and private dataset, does it mean that there is no need to discuss the privacy risk? Definitely not! Let's consider a scenario where a public dataset contains various images of cats, while a private dataset contains personal images of cats belonging to individual users. Although both datasets involve images of cats, the private dataset may contain users' personal information, such as their family photos or addresses. Therefore, even if the two datasets are similar in some aspects, the private data still carries privacy risks and needs to be properly protected. Taking Membership Inference Attack (MIA) as an example, consider an adversary that wants to probe a ML model to test membership of an individual's data in the model's training data. In this scenario, an adversary is more likely to have access to some representative images of the target individual, but not necessarily the ones used for training the model. As shown in Figure 8, we implemented the state-of-the-art LiRA algorithm in MIA. The experimental results demonstrate that our approach ensures the protection of sensitive information of the members in the clients' data (since the model training process has never been exposed to any private data). In contrast, traditional federated learning methods directly train on private data, posing a high risk of exposing the sensitive information of the members in the clients' data (i.e., for certain private data samples, attackers have a high confidence in identifying the client from which the sample originates). To the best of our knowledge, prior to our proposed approach, no one has put forth a training paradigm that effectively defends against LiRA while concurrently modeling utility (i.e., achieving high test accuracy).

• Finally, even for particularly challenging domains such as remote sensing images or fine-grained classification datasets, our method can easily adapt to these scenarios. We conducted experiments on several fine-grained image classification datasets, namely CUB-200, Stanford Cars, and also the satellite image dataset EuroSAT. CUB-200 is a challenging dataset consisting of 200 bird species, while Stanford Cars contains 16,185 images belonging to 196 classes of cars. See more details in the Appendix A.4.2.

Q4: does the proposed method work for federated learning on fine-grained classification such as CUB-200, Cars, and medical image datasets?

A4: Please refer to the table in A3.

We thank Reviewer HdAm for your valuable feedback and constructive comments. We have carefully answered all your questions and added extra experiment results in the following.

Q1: The proposed method may be highly dependent on the performance of both diffusion models and visual-captioning models. An ablation study of varying the foundation models is needed.

A1: Thanks for this insightful point. To investigate the impact of various generative models on the results, we followed the setting in [1]. Our experiments primarily focus on three prevalent conditional diffusion models: DiT[2], GLIDE[3], and Stable Diffusion. We use these off-the-shelf models to generate synthetic images. Specifically, for GLIDE and Stable Diffusion, the prompt was configured as "a photo of {label name}, real-world images, high resolution." For DiT, the input comprised the label ID corresponding to the ImageNet1k dataset. The images synthesized by DiT and GLIDE are of dimensions 256x256, whereas those produced by Stable Diffusion are of dimensions 512x512. As shown in the following table, even when we vary the foundation models used in our method, FGL consistently outperforms FedAvg by a significant margin. This observation serves as evidence for the generality of our approach. We have added these results in the Appendix A.4.3.

[1] Li, Zheng, et al. "Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?." arXiv preprint arXiv:2305.12954 (2023).

[2] Nichol, Alex, et al. "Glide: Towards photorealistic image generation and editing with text-guided diffusion models." arXiv preprint arXiv:2112.10741 (2021).

[3]Peebles, William, and Saining Xie. "Scalable diffusion models with transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Q2: Clients in federated learning are often assumed to have limited capacity in memory or computation.

A2:

• First, compared to FedAvg, our method introduces only one additional operation on the client side, i.e., prompt generation, which involves only forward propagation and does not impose significant computational costs. All computational operations are executed on the server side during the initial communication. The server trains a model with an excellent initial state, and subsequently, the client performs regular model updates, which means no additional cost compared with FedAvg.
• Secondly, our method is particularly well-suited for cross-silo FL, where the clients represent organizations or companies. In this context, the number of clients is typically small, but they possess substantial computational resources. Furthermore, this scenario emphasizes the importance of protecting clients' local data from potential leaks, which constitutes a significant contribution of our approach towards preserving privacy.

Dear Reviewer,

Thank you once again for your valuable comments. As the discussion stage is coming to a close in 2 days, we kindly request your feedback on whether our response adequately addresses your concerns. We would greatly appreciate any additional feedback you may have.

Thank you in advance!

Kind regards,

Authors