ZOOPFL: EXPLORING BLACK-BOX FOUNDATION MODELS FOR PERSONALIZED FEDERATED LEARNING

Thanks for your acknowledgment in our comprehensive experiments and intriguing! We see that your main concerns are on details of practicality, motivation, model privacy, and more experimental analysis. Now we answer them here.

1. The black-box FL setting lacks practicality.

Please refer to the general response for the problem setting.

2. The motivation for deploying zeroth-order optimization methods based on the local encrypted black-box model setup is not well-motivated. This setting implies that it is entirely possible to train an white-box emulator [1] as a proxy for the black-box model and directly perform first-order optimization based on the white-box emulator. However, the authors do not provide relevant discussions and experimental comparisons.

It is important to note that we are in a black-box setting while [1] uses a white-box setting. Thus, the comparison with [1] is neither fair nor meaningful. But we will cite [1] in the future.

Specifically, [1] involves compressing or substituting large models, requiring a deeper understanding, which differs from our current framework. We make the assumption that the internals of large models are entirely unknown. While training substitute models through obtaining outputs based on given inputs could be another avenue, as mentioned in our future work, 'Foundation models in ZOOPFL can be enhanced by other ways, e.g., auxiliary models, to serve as a complement to foundation models.' Currently, no one in the field has pursued this direction. Additionally, in our setup, each client has a limited amount of training data, potentially challenging the construction of robust substitute models ('For COVID-19, each client only has about 50 samples.').

[1] Xiao, Guangxuan, Ji Lin, and Song Han. "Offsite-tuning: Transfer learning without full model." arXiv preprint arXiv:2302.04870 (2023).

3. In terms of model privacy, the privacy leakage of a black-box model is closely related to the number of queries [2], but the authors do not provide theoretical or empirical studies on this.

A good question! However, our paper does not have such an issue. Why? The black-box models are in each client, and there is no data exchange between clients and servers, or between clients.

Regarding the privacy protection of the model, it's not within the scope of consideration for our current paper. Our focus lies in exploring how to better leverage a black-box model provided by the model suppliers—enhancing the alignment between local task inputs and outputs with the model.

4. The experimental section lacks ablation experiments with varying levels of noise added on the transformed data and visualizations of transformed data.

• First, we do have the experiments on varying levels of noise, as illustrated in Figure 6(b).
• As for visualization, we add such extra experiments in the revised version, as illustrated in Appendix F.5.

We hope your concerns will be resolved and the rating of the paper can be increased accordingly. Thank you!

After carefully reviewing the responses from other reviewers and the author, I have decided to keep my rating and lower the score for soundness and contribution as the author's response did not address my concerns.

The author highlights the main contribution of this paper as "Scenario Exploration." However, in my opinion, this new scenario may not be practical, and the exploration does not seem to be comprehensive.

Regarding the new scenario, a critical question arises: under what circumstances, with the foundation model available on the local client, do we need to set the model as an access-constrained black-box model? The motivation for introducing this specific "local black-box" scenario remains unclear. Typically, the use of a black-box restriction should be based on specific requirements, such as privacy protection needs or scenarios related to property rights, for instance, when the model serves as a prediction API or proprietary software. These situations necessitate essential related experiments on the number of queries (see black-box visual prompting [1] and black-box adversarial reprogramming [2]).

During the rebuttal phase, the author emphasizes this special "local black-box setting," focusing on the inability to perform backpropagation. If this is the case, there may be no need to emphasize the black-box setting. Instead, the paper could consider the adaptation of MeZO [3] to the Federated Learning (FL) scenario from the perspective of reducing computational and memory costs.

Furthermore, in the response regarding "why is it practical," the conclusion that "their proposed black-box FL is the only solution" is derived directly and confidently based on the high computation and communication costs of fine-tuning foundation models locally, which is perplexing and appears unrigorous.

Additionally, I noticed that the author selectively chose a comment from a reviewer, labeled the reviewer as "an emotional reviewer," and posted it on social media. I find this behavior to be inappropriate. Such actions, which disrespect the reviewer in active discussion, are truly disheartening and may diminish the enthusiasm of other reviewers for participating in discussions and collectively improving the paper.

[1] Oh, Changdae, et al. "BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
[2] Tsai, Yun-Yun, Pin-Yu Chen, and Tsung-Yi Ho. "Transfer learning without knowing: Reprogramming black-box machine learning models with scarce data and limited resources." International Conference on Machine Learning. PMLR, 2020.
[3] Malladi, Sadhika, et al. "Fine-Tuning Language Models with Just Forward Passes." arXiv preprint arXiv:2305.17333 (2023).

Thanks for your acknowledgment in our paper: the first to achieve federated learning with large black-box models and well-written and clearly structured! We see that your main concerns are on details of novelty, personalization, module functions, and concepts. Now we answer them here.

1. The idea seems very similar to the soft-prompt training [1], which is also working on the input surgery without directly inference the foundational model. What is the benefit of the auto-encoder pre-training in your paper?

We appreciate your recognition of the significance of [1] and we have cited this work in the revision. Where is the benefits of auto-encoder pre-training?

We use auto-encoders to alleviate the computational complexity associated with zeroth-order optimization. The rationale behind pre-training the auto-encoder stems from the need to adapt it to samples and tasks effectively. The pre-training step, as depicted in Figure 4 of the original manuscript, serves as a crucial aspect of our methodology, as illustrated by the ablation experiments.

In summary, we agree with your assessment and appreciate the opportunity to clarify the unique contributions of our work in addressing the where, how, and why aspects of input manipulation, particularly in the context of personalized federated learning with large black-box models. We will certainly emphasize the distinctions and innovations of our approach in the revised manuscript, giving due credit to the relevant literature, including Reference [1].

[1] Wang, Zifeng et al. “Learning to Prompt for Continual Learning.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 139-149.

2. What are the benefits of personalization? In the experiment part, it mainly focused on the overall accuracy boost compared to ZS, which does not reflect anything regarding to personalization. Even without step 3, this paper still makes its point regarding how to efficiently use the black-box model under FL setup.

Indeed, personalized learning stands out as one of the most crucial and practical challenges in FL [2], and our approach naturally tackles this issue. While we do furnish personalized outcomes for each client (as depicted in Figure 3 in the main text and Tables 4-6 in the appendix), constraints posed by page limitations compel us to emphasize average results in our report. It's worth noting that we explicitly stated in the original text, 'Our method achieves the best accuracy in most clients.'

Without Step 3 or implementing Step 3 via FedAVG, our method seamlessly adapts to the common federated learning (FL) setting. However, the incorporation of Step 3 not only enhances semantic matching but also leads to superior overall outcomes and personalized effects.

[2] Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. In The Eleventh International Conference on Learning Representations, 2023

3. What does ZS stand for in the paper? Does it stand for zero-shot training?

The meaning of ZS is introduced in the paper: ZS denotes "zero-shot pre-trained models", i.e., zero-shot inference using the pre-trained models. And with all due respect, the term "zero-shot training" in your question is incorrect since we cannot "zero-shot" train a model if there is no sample available.

4. I am not very clear why ZooPFL needs Semantic re-mapping. Could the author elaborate more on this?

The benefit of semantic remapping is introduced in the main paper. Here we explain it more. Short answer: the model outputs do not match the target task, so a re-mapping is needed.

Long answer: As highlighted in our main text, "Some popular language models such as BERT and GPT-2 in Huggingface utilize a random projection between extracted features and logits," and "This step endeavors to re-map logits into meaningful semantic spaces with a simple linear projection." The intentional inclusion of a random mapping followed by a specific semantic mapping is justified in practical terms. Moreover, when meeting data not covered by foundation models, foundation models can be able to extract meaningful features but be unable to map it to expected target labels. Semantic re-mapping eliminates these gaps. The ablation experiments further substantiate the significance of this component.

---

We hope your concerns will be resolved and the rating of the paper can be increased accordingly. Thank you!

Thank you for the response. But we are upset that reviewer still cannot understand our situation.

First of all, we are not dealing with cloud-hosted models but local ones on each client. The privacy concern makes this setting even more primary. Your understanding about this is correct. But this is not a minor setting.

Second, promptfl requires gradient to tune the prompt, which is not possible in the blackbox setting. As we discussed extensively in the related work part, prompt tuning is not possible.

We are glad that you read other review. But please also read other response to get a better understanding especially our general response.

Thanks for your acknowledgment of the paper: an important problem, helps us understand the scenario of incorporating FM in FL settings, and interesting contributions! We see that your main concerns are on details of privacy implication, more FM, larger task, and the memory footprint and computation complexity. Now we answer them here.

1. If we assume that the "black box" in figure 2.b runs externally, what are the privacy implications wrt to its input and output crossing the device boundary. If it runs on-device then what are the assumptions wrt to its size and the fact that it is a black box.

• First, we currently do not assume cross-device data exchange following existing work on PFL. So, the privacy issue does not exist.
• Second, for on-device training of our model, the sizes of models and data highly depend on the local computation hardware, but not our algorithm. By design, our algorithm supports all kinds of models and datasets.

2. The authors assume that the FM is a black box. With more and more FM being open-sourced, it would be great if the authors can further motivate their approach and what might be the main advantages of incorporating a black box.

Please refer to the general response for the detailed answer.

Short answer: even if you can have full access to the model locally, it is still expensive to actually train them. "Black-box" does not equal to close-source, even open-source models are expensive to train on ordinary devices.

3. The evaluation is mostly done on rather simple benchmarks. I was wondering if the proposed approach (to train just parts of the model) would carry enough capacity to tackle larger tasks. Maybe some discussion or even evaluation on a more complex task would be great.

In actuality, our study encompassed extensive experimentation across eight datasets that include two modalities: COVID-19, APTOS, Terra100, Terra46, SST2, COLA, Financial-phrasebank (Financial), and Flipkart. The results from these experiments underline the superiority of the framework proposed in our study.

Please be aware that the tasks we have selected are currently not adequately addressed or poorly performed by large models. Our aim is to align these large models with specific tasks, rather than undergoing retraining. Consequently, we do not require an abundance of data or a larger dataset.

4. The paper might benefit from some understanding of the memory footprint and computation complexity of this method. Overall, the main target of this method is to make FM training possible with FL (on-device). As a result, we should have a good understanding on the memory/computation overhead.

The answer is simple: we only introduce negligible memory and computation burden to the foundation models.

As depicted in Figure 4(d) of the original paper, it's evident that the parameters adjusted by our method are significantly smaller in scale compared to the large foundation models. This aspect reinforces the notion that our approach hones in on a smaller subset, effectively tailoring the model's adjustments, rather than necessitating the manipulation or direct handling of the entire foundation model. Our emphasis remains on the utilization of these large models as protected tools within federated learning, ensuring their functionality while mitigating the challenges associated with their training costs.

---

We hope your concerns will be resolved and the rating of the paper can be increased accordingly. Thank you!

Thanks for your acknowledgment in our research direction! We see that your main concerns are on details of practical scenario, novelty, and computational overhead. Now we answer them here.

1. In what practical scenario would a FL client host a foundation model, but does not have full access to it?

Please refer to the general response for a better understanding of our problem setting.

Short answer: The understanding of the word "black-box" could be broad. Black-box does not only deal with the case where one has no full access to the model; even if you have the model access, it is still extremely expensive to run model updates (Backpropagation) on the local devices given the increasing larger sizes of the models. As an example used in the paper, 'training GPT-2-small (Radford et al., 2019) requires at least two A100 GPUs for 16 hours, a resource unavailable to many'. We are a solution of using inference for BP to save computation and communication costs.

2. What are the key technical challenges and novel solution in this work?

Key technical challenges:

• efficiently update the foundation models locally
• distribution divergence in different clients

Our innovations to deal with them:

• A first pioneering exploration of the problem where models become too large to update, while FL clients still want to preserve privacy by not sharing data with the central server.
• A new framework to handle the problem using zeroth order optimization.
• We offer theoretical analysis along with extensive experiments to show the effectiveness of our work.

3. What is the additional computational overhead of the additional components (auto-encoder and semantic re-mapping) in the proposed method, when the full combined model is used for inference?

Simple: the additional computation costs are negligible compared to the huge cost of the foundation models. Quantitatively speaking, as depicted in Figure 4(d), the parameter volume of the head-tail adjustment module is several orders of magnitude smaller compared to the foundation model.

If you think these responses address your concerns, please consider increasing your score. Thank you!

Thank you very much for your feedback. We would like to briefly address your questions here, and we plan to upload a revised version in the next couple of days:

1. Experiments on costs.

In Figure 4(c), we mentioned the number of parameters, stating that fewer parameters imply lower cost under similar conditions.

2. The meaning of "ZS".

ZS means zero-shot and we have added the corresponding description.

3. The definitions of the quantities.

The definitions are in Eq. 41 on page 18 of the original paper.

4. How are s and r computed.

's' and 'r' represent the respective networks, and we use 'q' to denote the parameters to be calculated. 'r' can be optimized directly using SGD/ADAM, and we will make it clear in the revised version.

We greatly appreciate your detailed feedback, which has contributed to the improvement of our paper. If you have any additional comments or suggestions, please feel free to share them. Your insights are valued, and we look forward to further refining our work based on your input.

Thanks for the response. The authors claim that their proposed black-box FL has "low cost". However, I do not see anywhere in the experiments that measures this cost compared to baselines.

Furthermore, the writing of the paper needs substantial improvement. For example:

• The experiments use "ZS" as a baseline, but it is not clear what ZS stands for (I couldn't find it from the paper).
• In Theorem 1, I cannot find the definitions of the quantities $\Delta_\mathbf{u}$ and $\Delta_\mathbf{v}$.
• In Algorithm 1 (in Appendix C), $s_i$ and $r_i$ are part of the output, but the algorithm does not seem to include any step for computing $s_i$ and $r_i$. How are they computed?

Unfortunately, the more carefully I read the paper, the more I find the paper to be lacking scientific rigor. Even cleaning up all the definitions and other details requires a thorough and careful revision, besides other issues. It is clearly not ready for publication in its current form.

Thanks for the updates. For Table 3 containing the new results, could you explain why your method consumes less GPU memory although it includes additional processing? Do you use more CPU instead of GPU?

In addition, as mentioned in my earlier comment, it would be nice to compare the difference in cost between forward propagation only and forward+backward propagation, which is the key difference between zeroth order optimization and existing first-order based approaches. The current results do not seem to include this comparison. In other words, it would be nice to include some empirical evidence showing the advantage of zeroth-order optimization in terms of computational efficiency.