Federated Black-Box Adaptation for Semantic Segmentation

We thank the reviewer for the valuable comments.

1. Reproduction of training data:

   a) The clients can arbitrarily choose their networks. In the current setting, there is no dependence between the clients or the client and the server other than the output size of the client's last layer. The current work proposed a two-layer CNN as an example network.

   b) Even for a two-layer network, lets say an attacker knows the intermediate representations. Then the task for the attacker would be to find the input x and the function f given y where f is a small network and y is the leaked intermediate output. Now, to find out x, one could solve the equations y = f(x) and y’ = f’(x), where f’ denotes the derivative with respect to x. However, this method requires gradients to be known as well, which is not the case with out method.

   c) Furthermore, one method could be to learn a new network to produce the same outputs as the client models, given some input. In this case, the attacker would initialize a new two layer CNN and would have to query the original client model with learnable inputs. This way, they could train the new model with the client outputs as ground truths. However, for this case as well, the attacker would require access to the actual client models.

2. Sending Masks to the Sever: In the proposed work, masks are sent to the server for FOO. Even with this information, the attacker would require raw data to generate a new model that can emulate the client without gradients. For example, consider stable diffusion or similar methods that can recreate images given their masks. These would be based on public datasets and hence, would not be able to generate PII information that will be present in the raw data. In fact, given the mask, they can still generate synthetic data, but that cannot be considered as replicating raw data since it would still use a distribution of pixels similar to public datasets and not the private client data. We believe that not requiring model information transfer and gradient transfer is an important step in the direction of better privacy-preserving FL over existing methods.

3. Directly Splitting the Server Model: The network at server could also be split into two parts as mentioned by the reviewer. While this is a valid strategy, this would limit all the clients to use the same network. At the same time, the server would potentially have the complete model architecture information, which is not required in the current setting. The motivation behind proposing a new lightweight client model was that this would allow different clients to have their own design choices for the model with the condition that the output should have a particular dimension. Since the client is updated using ZOO, having a lightweight model might also help in getting a better performance.

4. FedBPT as added black-box method: We thank the reviewer for pointing out the FedBPT method. We will add it in the related work section. Please note that FedBPT was proposed to work with foundation models and as a prompting mechanism. Hence, it uses CMA-ES, a ZOO method to learn a prompt for the pretrained foundation model at each client and aggregates these prompts at the server. Since we want to learn the network parameters themselves and not the prompts, we cannot directly compare with this method. Instead, for the experiments, we tried replacing SPSA-GC with the CMA-ES method for updating the client in our algorithm. However, we see that it kills the program due to the large number of parameters in the network as compared to prompts. CMA-ES is an evolutionary algorithm that maintains a big group of candidate values for each trainable parameter, that is perturbed towards a better loss value in each iteration. Since the number of parameters is very large, we were not able to run CMA-ES as the ZOO algorithm.

Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you

We thank the reviewer for the valuable comments.

1. Reproduction of Training data: As the reviewer suggested, the intermediate representations and segmentation masks can be used along with diffusion models. However, to train these models, one would still require the raw data to act as ground truth labels during training. If one were to use public datasets for training such diffusion models, the diffusion model would still learn to generate images from a similar distribution to the public datasets. This is not the same as regenerating raw data since raw data can have personally identifiable information (PII) that won't be learnt by diffusion models which are trained on public datasets. In such scenarios, in fact, the output from Stable Diffusion can be considered as a good source for synthetic data, which looks similar to raw data but would arise from the public data distribution. In fact, some recent works also highlight the property of stable diffusion and other image generation methods to copy the data they were trained on [1-4]. In such a scenario, in the absence of raw data from the client, the best a diffusion model would be able to do is to generate images from public datasets similar to the segmentation mask, which we think cannot be considered as reproduction of client data. We believe that not requiring model information transfer and gradient transfer is an important step in the direction of better privacy-preserving FL over existing methods.

[1] Frame by Familiar Frame: Understanding Replication in Video Diffusion Models

[2] Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models

[3] Analyzing bias in diffusion-based face generation models

[4] Replication in Visual Diffusion Models: A Survey and Outlook

2. Adding more Comparisons: FedSeg is a method that is built upon FedAvg and works by sharing model weights across server and client. Hence, this would act as an upper bound for our method, just like FedAvg. We verify this for CAMVID and Cityscapes and add it to Table 1 presented in the extra table page for rebuttal. We see that for both the datasets, it generally performs better than FedAvg. Please note that this cannot be directly compared with BlackFed since our approach operates in a more restricted setting where model sharing is not allowed. We also add other methods like FedPer in this table, as suggested.

3. Novelty for ZOO: Please note that we do not claim novelty for the ZOO method used. In fact, we cited the BlackVIP paper as being the work that introduced SPSA-GC as a ZOO method (ref. No. 38 in the paper). The major contributions of our work include formulation of the FL problem in the blackbox setting using split networks to allow gradient-free updates. The proposed approach formulates the FL problem using a lightweight client and a parameter-heavy server, along with a round-robin algorithm that can allow SPSA-GC to work well, since it was originally meant to fine-tune pre-initialized foundation models. The proposed approach shows that by alternating between clients, and updating them with ZOO and FOO, one can achieve good performance. In addition, we identify the catastrophic forgetting problem and come up with a solution using hashmaps.

4. Inference: During inference, uploading the features to server would increase time as pointed out. Some ways to reduce this would be to upload batches of features at the same time that can reduce the amortized time cost. An alternative is to get a copy of the server initialized with the hashmap weights for the client after training is complete. Then, client can perform the inference locally. This assumes that the client has sufficient compute power. Note that the entire hashmap need not be shared, but only the entry corresponding to the client.

5. Additional Method FedPer: We thank the reviewer for pointing out this method. We will add it to the related work and comparisons in the revised paper. In addition, we added results with FedPer in Table 1 in the extra rebuttal page. However, FedPer does not use split networks. Instead, it is similar to FedAvg, the difference being that not all weights are shared with the server. Here, the majority of the weights are shared with the server and aggregated. The remaining weights are "personal" to the model and allow for better local performance. These are not shared with the server. In contrast, there are no weights shared between the client and server in Blackfed. The output from the client is given to the server, where it is further processed, making this a split network. Furthermore, The FedPer paper only performs the task of classification. In order to adapt this for segmentation, we considered the weights of the classifier head of DeepLab v3 as personal weights and the rest of the backbone weights were averaged in the server. We find that in most cases, this improved local performance over FedAvg, but reduced OOD performance.

6. Ablations with more architectures: Table 3 in the paper compares performance with three different architectures for the server. This includes Unext and Segformer in addition to DeepLab. Segformer is a transformer-based method while the others are CNN-based.

Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you

We thank the reviewer for the valuable comments.

1. Comparison with the baseline from papers with code: The dataset of Cityscapes has data from 18 different centers. In the original splits of Cityscapes, the training data contains images from some of these centers, while the data from rest of the centers is in the validation and testing sets. In our case, we want each center to have its own training, validation, and testing data. The goal of the Federated Learning model would be to do well on test data from other centers in addition to doing well on the in-house test set, indicated by the “OOD” and “Local” columns in the tables. Hence, the data splits are done differently. Training DeepLab v3 on single machine is the same as “Combined training” in Table 1, where all the training data is merged to train a single model, that achieves a DSC of 0.77, which is close to the number shown on PapersWithCode. This represents an upper bound to what BlackFed can achieve since training on a single machine means that it has access to data from all centers.

2. Clarification on Trend of Model Complexity vs Performance: The model complexity includes the number of trainable parameters which increase from UNext to DeepLab to Segformer. But in table 3, row 1, we see that the individual performance increased for DeepLab but decreased drastically for Segformer, even though it has the most number of parameters. Since the data is limited in this case, we see that increasing the number of trainable parameters beyond a certain point causes the validation performance to suffer. However, for the case of combined training, the data quantity is higher, so val performance increases from UNext to DeepLab v3 to Segformer as expected. This is the desired trend in the FL algorithm as well since it also has access to more data than the limited data case of individual training. This trend is seen in BlackFed as the performance increases from Unext to DeepLab and does not drop when the model complexity increases from DeepLab to Segformer.

3. Reason for Lower Resolution: We wanted to do the training of a given client on a single GPU, since institutes like medical centers do not have a high compute power. Hence, we downscale the image without affecting its aspect ratio. Training on a larger resolution would require much higher computation. Since we are emulating all client and server computations in the same compute center, this would require increased compute, especially in the case of 18 centers like Cityscapes. However, we plan to release a model zoo on Github with pretrained checkpoints at different resolutions since training with larger resolutions can improve performance.

4. Clarification on Server Architecture: All the parameters in the approach are trainable and there are no frozen layers. In our experimental setup, we use a two-layer Convolution Net in the client to reduce the load on each client. The output of this layer is a feature map of the shape H X W X 64. This is transmitted to the server for further processing. Hence, the first layer of the server architecture should expect an input with this shape. We are using DeepLab v3 architecture for the server. However, instead of starting from the Conv1 layer in DeepLab, we are starting from Conv2 (and deleting Conv 1 from the network), which also expects an input with 64 channels. Hence, in order to continue the training in the server from the client's output, we designed the server architecture in such a way. In other words, this can be considered as one network, the first two layers of which are in the client while the rest of it is in the server, hence the term “split network”.

Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you

We thank the reviewer for the valuable comments.

1. Low Performance on Polypgen: This case indeed serves as an important application area. Here, for 4 out of 6 centers (C1, C2, C4 and C5), it is beneficial to use the FL method. But for C3 and C6, the OOD performance decreases. This could be due to other centers dominating the training, thus causing catastrophic forgetting for particular centers like C6. But as pointed out by the reviewer, this should have been a problem elsewhere as well. In such cases, maybe the learnt weights are not good due to a more adverse distribution shift between centers in Polypgen as compared to CAMVID. In such cases, it would be important to control the training of the FL algorithm to make it more robust, which is an interesting and important direction for future research in this area.

2. Counter-Intuitive Performance of Whitebox on ISIC: As pointed out by the reviewer, for ISIC, the whitebox method strangely performs subpar. We verified this through various runs and changes to the hyperparameters. This anomalous behaviour may be due to the model getting stuck in some local minima during training because of overparametrization.

3. Comparing Individual Performance for CAMVID and Cityscapes: In the Blackfed case, for both datasets, the server model benefits from the entire dataset, hence the simliar performance. However, in the individual case, the model performs subpar in some of the clients. For CAMVID, individual performance of C1 is much lower than others. But for Cityscapes, there are some clients with a good performance and some clients with subpar performance. Hence, since the number of clients is more, on an average, the indivual performance is lower. This also depends on the choice of model architecture and data distribution since the number of data points is so less.

4. Additional Memory Overhead: As the reviewer pointed out, the proposed approach comes with an added memory overhead for storing the hashmap, which scales with the number of clients. However, the motivation for the method was based on the assumption that while the client has limited memory, the common server can have a large memory. Based on the model architecture, each entry in the hashmap would be of the order of 100 MB or less. If there are 100 clients, this would translate to 10 GB, which is a realistic number. For larger models, we believe that the memory overhead cost would not be a bottleneck in comparison to the cost of training such models. One interesting direction in such cases would be using Parameter Efficient Finetuning Methods (PEFT) like adapters or LoRA and only storing them.

Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you