On the Byzantine-Resilience of Distillation-Based Federated Learning

Thank you for your feedback, please let us address your concerns in detail.

The paper only considers iid setting. Although the authors claim that non-iid setting in FL poses serious issues (line 122), they don't specify what kind of issue makes it unsuitable for KD-based FL. Moreover, non-iid setting is essential in FL and is closer to realistic scenarios.

We do not claim that non-iid data is specifically problematic for KD-based FL as opposed to FedAVG. While the Byzantine-resilient FedAVG literature is extensive, unfortunately, research on Byzantine-resilient FedDistill is currently limited. Although addressing the non-IID case is a valuable direction, it is significantly more complex to provide any meaningful advancements in the non-IID byzantine setting. Up to now, for FedDistill, only few works have discussed the IID byzantine setting, which we focus on. It is essential to first gain a clear understanding of the IID setting. This approach mirrors the development of the Byzantine-resilient FedAVG literature, where focusing initially on IID data established a foundation for later addressing non-IID challenges.

The scale of experiments is limited. The paper only considers cross-silo settings (a small number of clients) and lacks cross-device settings (over a hundred clients).

KD-based FL methods struggle in cross-device settings with standard ML datasets. As the number of clients increases, the fixed dataset size means less private data per client, making initial model training challenging. This issue is particularly problematic before the first communication round. We have addressed this in Appendix C with ablation studies.

The ExpGuard assumes that the Byzantine clients are identical during training, which may be not the case in reality.

This assumption is standard for defence methods that rely on historical information, such as (Alistarh et al.,2018; Allen-Zhu et al., 2021; Karimireddy et al., 2021). Without this assumption, one cannot use historical information. In fact Karimireddy et al., 2021 show that for FedAVG-type algorithms, convergence is impossible in this scenario, see their Remark 3.

In line 328 and figure 2, shouldn't the least likely class be dog rather than cat?

Thanks for pointing out this mistake, we have fixed it.

CPA attack requires a pretained model to obtain the similarity matrix, how do you get it in the experiments? Is this effective in a more realistic setting?

In the experiments, we used publicly available models pretrained on the datasets in question. We then ran inference on the public dataset and computed the covariance matrix between the class probabilities of those predictions. We will add a more detailed description to the appendix.

While we have noted in the paper that having a pretrained model is not necessarily a realistic assumption (lines 347-350), our goal was to search for an attack that would be able to considerably perturb the training process. As one can see from the experimental results, even under the strong assumption that one has access to a good similarity matrix, the impact of CPA stays limited. A more realistic approach might be to use prior knowledge about class relationships or simply use the server model from the last round. But we expect these methods to work at best a good as using a pretrained model for obtaining the similarity matrix.

Will the distribution shift between clients' data and public data affect the performance of the server's model? It's better to provide additional experiments or theoretical analysis on how different levels of distribution shifts between client and public data might impact the server model's performance. This could help readers better understand the limitations and robustness of the proposed approach.

Indeed, you are correct, a significant difference between clients' data distribution and the public dataset distribution will hurt the performance of KD-based FL. While KD-based FL certainly provides a lot of advantages over methods that share parameters, this is indeed an issue, when relying on a public dataset, as such algorithms do. However, KD-based FL is widely used and the goal of this paper is to understand the impact of byzantine clients in this setting. Please let us note, that we are not advocating KD-based FL as a better method than anything else. Our work is constrained to the byzantine analysis.

We thank you for your feedback. We hope to have addressed all your concerns. please let us know if you have further comments that help us to improve the paper.

We thank you for your detailed review. We will address your concerns in detail, hopefully resolving some misunderstandings.

Weaknesses

The most obvious weakness of the paper is that it addresses the IID data distributions. In contrast, the current state-of-the-art (SOTA) Byzantine resilient federated learning methods focus on non-IID data setups.

While the Byzantine-resilient FedAVG literature is extensive, unfortunately, research on Byzantine-resilient FedDistill is currently limited. Although addressing the non-IID case is a valuable direction, it is significantly more complex to provide any meaningful advancements in the non-IID byzantine setting. Up to now, for FedDistill, only few works have discussed the IID byzantine setting, which we focus on. It is essential to first gain a clear understanding of the IID setting. This approach mirrors the development of the Byzantine-resilient FedAVG literature, where focusing initially on IID data established a foundation for later addressing non-IID challenges.

In Section 4, the motivating example to compare the vanilla FedAvg algorithm and the FedDistill algorithm is rather unfair. This is because the adversaries for the vanilla FedAvg employ model poisoning strategies, while the ones in FedDistill use data poisoning.

We note that byzantine clients cannot poison the model in FedDistill, as they can only influence the server model by modifying the labels on the public dataset, i.e., data poisoning. So the unfairness you pointed out is precisely what makes FedDistill so resilient. We refer to Lines 219-244 for a more detailed discussion of how the threat vectors of byzantine clients differ between FedAVG and FedDistill.

In Section 4, the presentation in the byzantine resilience of FedDistill can be improved. All the theoretical theorems and lemmas have been deferred to the Appendix, making it hard to appreciate the real significance of resilience.

Thank you for your suggestion. We will highlight our theoretical results more prominently in the final revision of the paper.

In Section 5, the loss minimization attack (LMA) is a bit contrary to the prior Byzantine federated learning literature, e.g., [Karimireddy et al. (2022), Allouah et al. (2023)], where they assume the server is curious but honest. Here, the server is not reliable in the sense that it can manipulate the generated model parameters to mislead the training. At one extreme, the server can completely replace the model parameters to destroy the whole training process.

We are not entirely sure whether we correctly understand your remark. LMA is an attack that byzantine clients perform, to find adversarial predictions, which are sent to the server in order to negatively influence the training process. The challenge for the server is to reduce the impact of byzantine predictions, to eventually learn a good model in spite of the byzantine clients. Could you elaborate how this is related to the server being "curious but honest"?

In Section 5.2, the authors did not explain why ExpGuard works, except by citing [Littlestone & Warmuth, 1994]. [...] In Section 5, can the authors further justify ExpGuard? In particular, why does the exponential weight work, but the other choices won't?

The goal of the server is to discard the byzantine predictions and to only use the predictions of the honest clients. In order to have an impact on the aggregated predictions, the byzantine clients must consistently make predictions that differ from those of the honest clients. Therefore we expect that the predictions made by the byzantine clients are on average further away from the mean honest prediction, which the server does not know. However, we know that robust aggregation methods such as the GM reduce the impact of byzantines and are hence close to the mean honest prediction. Hence, ExpGuard takes this robust aggregator as a proxy for the honest mean predictions and tracks how much each client differs from it on average. Specifically, this difference is quantified by the outlier score, i.e., the sum of distances from the robust aggregator over all samples. We expect clients that have a higher outlier score, i.e., that are on average further away from the proxy, to be more likely to be byzantine and vice-versa. ExpGuard assigns a weight to each client based on this outlier score via exponential weights update.

We thank you for pointing out this gap in our explanation, we will add it.

It is recommended to group HIPS together with LMA & CPA and present the defending strategy after the attacking strategies.

Thank you, we agree that this could be improved, we will change the order of presentation in the final version of this paper.

Thank you for your feedback. We will address your concerns in detail.

It is not clear what is the strength (mean/variance) of the Gaussian attack that is used to show the superiority of KD-FL against FedAvg.

Thank you for pointing this out. The attack consists of sending $\mathcal{N}(0,1)$ noise instead of the updated weights. We will add this information.

Meanwhile, it is not fair to draw the conclusion that KD-based method is more resilient to FedAvg by one comparison between random Gaussian attack to FedAvg and the random label flipping for KD-FL. A fairer comparison could include some other attacks.

Please let us clarify, that we do not claim that FedDistill is in general more byzantine-resilient than FedAVG. The goal of this article is to examine the byzantine-resilience of KD-based FL algorithms as these methods are becoming more popular, not to show that either FedAVG or FedDistill are more byzantine-resilient. Both methods have their advantages and drawbacks. Figure 1 is a motivating example showing that it is easy to find simple attacks that completely prevent the server from training a model in vanilla FedAVG, while in FedDistill such simple attacks have very little impact.

The proposed ExpGuard+F method is similar as the FLTrust methods, which is okay, however, the strength of the proposed method is not exactly reflected from the experiments. For example, is ExpGuard+F better than ExpGuard+GM and if yes, why? [...] ExpGuard shares some similarities with the FLTrust method, which is not new. It would be better to clarify the strength and difference of ExpGuard against FLTrust [1].

Thanks for pointing out this article, we were not aware of it and will add it to the related work. FLTrust differs from FedDistill in two key aspects. First, the approach of FLTrust is specific to FedAVG-type methods meaning that the analysis and intuition does not carry over to FedDistill. Second, while ExpGuard uses only the predictions of the clients on other samples and from previous rounds, i.e., information that is available anyways, FLTrust relies on an additional labeled dataset available only to the server in order to compute a trust score. This is a much stronger assumption.

We discussed the performance of combining ExpGuard with different robust aggregators in lines 402-407 and provide a experimental results in Table 5, Appendix C. There, we see that ExpGuard+F significantly outperforms ExpGuard+GM, among others.

The study of HIPS is interesting, however, the result simply suggests that sometimes robust aggregation is worse than mean aggregation. To make this point more valuable, it might be better to have a trade-off analysis or provide some guidelines to how to choose robust aggregation methods.

The point we want to make is that while the resilience of EG+F might be slightly worse than that of the mean against HIPS attacks, mean aggregation is not resilient at all against attacks without HIPS. Therefore, if the server does not know what kind of attacks to expect, it is still preferable to choose EG+F as it will maintain a good worst-case performance at the cost of a slight decrease in performance against HIPS attacks.

The prediction for honest agent 4 in Figure 2 should be [0.61, 0.08, 0.31] to ensure the correct average prediction.

Thank you for pointing out this mistake, we will fix it.

At least GM has been widely used in FedAVG for robust aggregation, therefore in line 387, when the authors say “As a result, applying them to aggregated parameters in Byzantine FedAVG is often impractical” does not make sense to me. It would be great if the authors either provide evidence supporting their claim about impracticality, or to revise their statement to better reflect the current state of practice in the field.

Thank you for pointing out this inaccuray. What we intended to say was that these methods can lead to a considerable computational overhead when applied to the parameters of large models, whereas the computational cost of aggregating predictions with these methods is negligible. We have changed the writing accordingly.

We thank you for your remark. Let us address your concerns in detail.

the formal guarantees (left to appendix D) deserve a more prominent place in the paper, given that the core contribution of such a paper is to formally derive security guarantees and their limits when (in this case) the space of all possible attacks cannot rely on empirical validation

Thanks for that remark, we appreciate that you think our theoretical results should be highlighted more prominently. We agree. However, at submission, we thought that the readership of ICLR might potentially be more interested in empirical results. Yet, we have provided these theoretical results in the appendix, so if we understand correctly, your concern is mainly a matter of presentation? We will make the theoretical results more prominent in the final revision of the paper.

What would be the effect of most comon robust aggregation schemes on KD-based FL? how would that change the bound in appendix D.1?

Most commonly used robust aggregation methods such as the coordinate-wise trimmed mean or the coordinate-wise median cannot be used to aggregate predictions, as the aggregated vectors do not necessarily lie in the probability simplex, see Appendix D.3 for a simple counter-example. This however is a necessary condition for a prediction vector to be meaningful.

For aggregation methods that produce vectors within the probability simplex and have guarantees independent of statistical assumptions, we can tighten the error bound. For instance, using Allouah et al. (2023), we can bound the difference between the honest mean and the GM.

We thank you again for your feedback, please let us know if you have further comments that help us to improve the paper.