Adversarial Training for Defense Against Label Poisoning Attacks

We thank the reviewer for their comments. We are glad you found the FLORAL framework interesting and recognize its theoretical backup for its design. We address your concerns and questions below.

• [Baselines & Q1] We first need to clarify that the reference [2] you provided, is already considered a baseline in our experiments. Please note that the baseline 4. LS-SVM (line 374, Section 4) is exactly the same method given in [2]. We cited their arXiv paper, however, we changed the citation to the one you provided in our updated manuscript. Prompted by your feedback, we further compared FLORAL against the methods [1,3], with results presented in Appendix J of the updated manuscript. As shown in Tables 11-14, the FLORAL approach significantly outperforms these baselines, with up to 40% improvement in accuracy level in most of the datasets. (e.g., in Table 14, approximately 45-50% improvement against RSRNN [3]; in Table 12 up to 17% improvement against RS method [1], etc.).

• [Q choice& Q2] Q should be set as such because the attacker (inner problem) outputs a poisoned label set $\tilde{y}$, which the model (the outer problem) learns a soft margin SVM classifier using the poisoned label set $\tilde{y}$. Hence, compared to the classical formulation for a soft margin SVM classifier (formulation provided in Section 2 lines 117-121) (Boser et al., 1992; Hearst et al., 1998), the label set contains potentially poisoned labels. Because the features are not poisoned, the kernel matrix stays the same as $K_{ij}$ with the classical formulation for a soft margin SVM classifier.

• [Using dual variables & Q2] Using SVM formulation and its dual variables directly provides us with an interpretable framework to identify influential training points. The dual variables explicitly quantify the contribution of each data point to the decision boundary. Influence functions often require approximations, by computing gradients or may require assumptions such as smoothness [4], which is eliminated when dual variables are used. In our paper, we only claim that using dual variables has the same intuition as using influence functions.

• [Generalizability & Q3] As we already demonstrated in IMDB experiments with RoBERTa + FLORAL integration, FLORAL can be integrated with other model architectures, directly by using it as a robust classifier head. That is, extracting the last layer embeddings from the model e.g. RoBERTa or a neural network, etc. (which will be $x_i$’s in the algorithm), and then training with FLORAL. As shown in IMDB experiments, FLORAL integrated on top of complex model architecture such as RoBERTa achieves better robustness and supports the adaptability of the FLORAL framework in integration with other model architectures. We think that it is an interesting way to highlight FLORAL's effectiveness and usefulness in modern real-world classification scenarios, where FLORAL directly improves the robustness of some underlying complex model architectures (such as a pre-trained language model in the case of RoBERTa) on classification tasks. To demonstrate the integration further, we performed additional experiments on NN + FLORAL integration on the Moon and MNIST datasets. As given in Appendix-L in the updated manuscript, with FLORAL integration, NN achieves a higher robust accuracy level, e.g. see Figure 20-(d) that FLORAL provides approximately 10% improvement on the accuracy.

Overall, the additional experiments have added depth to our empirical analysis, and further demonstrated FLORAL’s effectiveness against a diverse set of baselines and generalizability with other model architectures. We are happy to expand further if any parts remain unclear.

[1,2,3] are given in the reviewer’s comment.

[4] Koh, P. W., & Liang, P. (2017, July). Understanding black-box predictions via influence functions. In International conference on machine learning (pp. 1885-1894). PMLR.

• [Comparison with RoBERTa] The experiments with RoBERTa show the adaptability of the FLORAL framework in integration with other model architectures, and how it improves the robustness of those models against the label poisoning attacks carefully crafted for such models. This highlights FLORAL's effectiveness and usefulness in modern real-world classification scenarios, where FLORAL directly improves the robustness of some underlying complex model architectures (such as a pre-trained language model in the case of RoBERTa) on classification tasks. We do not agree with the reviewer's comment that attacks should not be targeted toward the RoBERTa model. Because safeguarding against targeted attacks is indeed highly relevant for the application scenario we have in mind. Based on your comment, we further clarified the aim of this experiment in the text and updated our manuscript accordingly (lines 463-464).

• [Label poisoning statement & Q5] When we consider classification accuracy as the measure, then given (fixed) a classifier $f(x) \to y$, flipping the label of any data point, i.e. $y$ to $\tilde{y}$, would contribute to the accuracy by -1. (As x is the same, our fixed classifier assigns the label y). Whereas, in a feature attack case: $x \to \tilde{x}$, the classifier $f$ can assign a different label depending on the perturbed $\tilde{x}$. Hence, the effect on the accuracy is not uniform.

• [Statement for the minimax formulation & Q5] In the case of a minimax formulation with Hinge loss, the attacker creates the worst possible attack by focusing on the points farthest from the decision boundary on the wrong side, as these would incur the highest loss. However, in a soft-margin SVM, the decision boundary is determined by the support vectors, which are typically closer to the boundary. These farthest-point attacks are unlikely to have a substantial impact on the boundary itself as soft-margin SVMs are shown to be robust to outliers (see [4] Section 1.5)

• [White-box vs capability statement & Q5] Having read the statement with your comment, we agree with the reviewer. We have updated the statement as “the attacker’s knowledge of the training data…”, since adding a budget implicitly simulates the case when the attacker cannot alter all of the training labels.

• [Q3]: In Algorithm 2, the resulting $\mu$ is an approximation to the optimal one (for large-scale settings), hence, the constraint (6) may be satisfied with some error. This reflects the global convergence as an additional $\frac{\epsilon}{1-\kappa_{\lambda}}$ term to the right-hand side of the Equation (33) of the updated manuscript, where $\epsilon$ is the error amount. Thus, the global convergence guarantee still applies for approximate $\mu$ (provided that $\delta'$ is adjusted accordingly and that the approximation error in $\mu$ is bounded).

Overall, we thank you for the constructive feedback. With the additional analysis we provided, we extensively analyze FLORAL’s performance across a diverse set of baselines, against a diverse set of label attacks and on different types of datasets.

We are happy to expand on the above points further if further questions remain.

[1] Xiao, H., Biggio, B., Nelson, B., Xiao, H., Eckert, C., & Roli, F. (2015). Support vector machines under adversarial label contamination. Neurocomputing, 160, 53-62.

[2] Andrea Paudice, Luis Muñoz-González, and Emil C. Lupu. (2019). Label Sanitization Against Label Flipping Poisoning Attacks. In ECML PKDD 2018 Workshops.

[3] Robey, A., Latorre, F., Pappas, G. J., Hassani, H., & Cevher, V. (2024). Adversarial training should be cast as a non-zero-sum game. International Conference on Learning Representations (ICLR'24)

[4] Schölkopf, B. (2002). Learning with kernels: support vector machines, regularization, optimization, and beyond.

Thank you for your thoughtful feedback. We are happy that you recognize the originality and novelty of our proposed framework, as well as appreciate the theoretical analysis and presentation provided in our paper. We address your concerns and questions below, including additional experiments and paper edits inspired by your comments.

• [AT defense & Q1] As noticed by the reviewer, when the training set is clean, adversarial training (AT) takes into account the worst attacks created by the attacker using the clean training set. Using these adversarial examples, the model is trained to avoid overfitting and to reduce the sensitivity to a small set of critical training points, which improves robustness. Thus, the FLORAL approach directly reduces the sensitivity of the model output on single training points, which, as can be argued from the perspective of algorithmic stability, helps generalization. When the training set is already poisoned, FLORAL is effective in identifying already poisoned labels. Indeed the reviewer’s intuition about FLORAL disrupting the initial attack is correct. To further demonstrate this with a quantitative analysis, we provided the percentage of recovered poisoned labels by FLORAL in Figure 19, and a discussion in Appendix K of the updated manuscript. As shown, FLORAL, on average, disrupts 30% of the initial attack, while only (erroneously) affecting a small number of unpoisoned data points. We evaluated this over different hyperparameter choices and showed that FLORAL consistently achieves this rate. We also demonstrate an example trace of a round in Figure 19, illustrating which points are initially poisoned and which of them are recovered by FLORAL on the Moon dataset.
• • Also, note that we have already shown a similar quantitative analysis with IMDB experiments, see Figure 10-(a) in Appendix D.2, that FLORAL maintains (on average) >50% of the influential training points. When the training set gets more adversarial, the recovery rate decreases naturally. However, even in the most adversarial setting with 40% adversarial labels, FLORAL maintains >50% of the influential training points for the model’s predictions. Compared to Figure 10-(b), the RoBERTa model not integrated with FLORAL shows a significant change in the influential training points. This clearly demonstrates that FLORAL diminishes the impact of already poisoned samples.

• [Non-zero-sum Stackelberg game formulation] Our formulation relies on the following key points (which we explain in Section-3):
• • As shown by Robey et al. (2024), the min-max formulation for AT does not capture the attacker’s objective correctly, since from the optimization perspective, the attacker should maximize a lower bound of the loss function. This motivates the non-zero-sum formulation.
• • Our formulation is a Stackelberg game since the attacker generates the label attack using the dual variables of the model. The interplay starts with initial model parameters $\lambda$, and then the attacker takes action $(\tilde{y}(\lambda))$ based on observed model parameters $\lambda$. This creates a leader-follower type of game (see Figure 3 which provides a graphical summary of this interplay).

• [Equilibrium & Q2] The equilibrium stands for the best-case outcome for the defender when facing the worst-case poisoning attack. At equilibrium, the attacker cannot find a better label attack without worsening the defender’s response, and vice versa.

• [Additional attacks & Q4] We further evaluated FLORAL defense against alfa-tilt [1] and LFA attack [2] from the literature and report the results in Appendix H-I. These analyses further support FLORAL’s effectiveness against diverse attacks from the literature.
• • Some of the baselines (e.g. K-LID (Weerasinghe et al., 2021), LN-SVM (Biggio et al., 2011) )) indeed consider attack rates 30%-40%. FLORAL consistently outperforms or is on par with all baselines, except for the alfa attack for smaller attack rates. For larger attack rates FLORAL significantly outperforms all baselines (including alfa), while many of the baselines fail.

• [Additional dataset & Q4] We provide extensive additional experiments on the MNIST dataset in Appendix-G. We also experimented on MNIST using the alfa-tilt attack (see Appendix-I). These analyses showed that FLORAL achieves higher robust accuracy compared to baseline methods (typically improving by e.g. 4.5% against the next best method in the most adversarial setting (see Figure 18-d)). We now clearly show FLORAL generalizes well across a diverse set of datasets from numerical, text, and vision contexts, under different label poisoning attacks.

We further address your questions below.

• [Label poisoning statement & Q5]: Under a perfect classifier (tries to achieve 100% classification accuracy), the effect of a label attack on the classification accuracy would be +1 or -1, depending on whether the point is already poisoned or not, however, still irrelevant where the point is located (uniform over the location relative to the decision boundary).
• [White-box vs capability statement & Q5]: We safeguard against the worst-case attacker and assume that this attacker has access to the labels of the B support vectors, which creates limited access to training points. In each iteration of FLORAL, the attacker has access to B support vectors, from which it can poison $k$ (with limited capability). With our statement, we aimed to only “mimic” the scenario in practice.
• [AT defense & Q1]: The percentage of recovered poisoned labels quantifies how many of the poisoned training points are flipped back to their original label by the FLORAL framework. For instance, in the situation where 5% of the data points are poisoned, FLORAL is consistently reversing the labels of 30% of these, which means that at each iteration FLORAL is much more likely to revert the labels of poisoned data points than to erroneously flip the labels of unpoisoned ones.

We hope that these points clarified your questions.

Dear Reviewer t4Fm,

As we approach the end of the discussion period, we would like to ask if there is any additional information we can provide that might influence your evaluation of the paper. We believe our previous response with additional experiments conducted & paper edits prompted by your feedback, has effectively addressed your initial concerns and clarified some points that may have been overlooked.

Thank you once again for your thoughtful consideration of our work.

We thank the reviewer for the comments. We are glad you appreciate the presentation, writing, and performance of our approach. We address the raised concerns and questions below:

1. [Baselines] The most recent baselines addressing defense against label poisoning indeed date back to 2021-2022, a fact also pointed out by reviewer jdAy (see the corresponding references). We further compared our approach against recent methods [1] and [2], with results presented in Appendix J of the updated manuscript. As shown in Tables 11-14, the FLORAL approach significantly outperforms these baselines, with up to 40% improvement in accuracy level in most of the datasets.
2. We changed the figure and table placement, particularly for pages 9-10.
3. We compared FLORAL against 8 diverse baselines + 2 additional provided [1,2], ranging from fundamental approaches to state-of-the-art defense methods.
4. As stated in our contributions in Section 1, to the best of our knowledge, this is the first work to introduce adversarial training (AT) as a defense for label poisoning attacks. Hence, there is no further AT method that specifically addresses defending against label poisoning attacks. We already included and compared our method against an AT method (NN-PGD) which defends against feature perturbation attacks, we think adding extensive comparison against other AT defense feature perturbation methods is beyond the scope and not aligned with the main problem we address in our paper, which is a defense against label poisoning attacks.

• Q1: We used the reference citation format (“\bibliographystyle{iclr2025_conference}”) in our paper. Presumably, the coloring was the concern (which we would find somewhat surprising though?), we changed the coloring to the usual color in the updated manuscript and hope this clarifies the issue.
• Q2: Our approach is indeed not restricted to binary classification tasks and can be extended to multi-class classification settings using a one-vs-all kernelized SVM strategy. To ensure clarity, we have already provided Algorithm-3 in Appendix-E. Given the straightforward nature of this extension, the details we presented provide a sufficient foundation for applying our approach to multi-class classification.
• Q3: The Moon dataset is a 2-dimensional synthetically generated dataset (illustrated in Figure 7 in Appendix C.1). We created train/val/test splits for 5 replications and for each setting, we generated adversarially labeled dataset by flipping the labels of points farther from the decision boundary of a linear classifier trained on the clean dataset (see Section-4). For the IMDB dataset, we extract embeddings (768-dimensional numerical representations) for the data points using the RoBERTa language model. We first fine-tune RoBERTa model on the IMDB dataset. Then, we extract the last layer embeddings of the fine-tuned model which serve as embeddings for the data points in the IMDB dataset.
• Q4: Soft-margin SVMs are shown to be robust to outliers (see [3] Section 1.5), however, as the dataset gets more adversarial (see 25% setting), the influence of individual patterns (adversarial examples) gets more severe. Hence, the reviewer’s observation no longer consistently holds. Also, as given in additional results in Appendix D.1 and Appendix G, Vanilla SVM does not consistently perform well in all settings.
• Q5: The experiments with RoBERTa show the flexibility and adaptability of the FLORAL framework in integration with other model architectures, and how it improves the robustness of those models against label poisoning attacks. This highlights FLORAL's effectiveness and usefulness in modern real-world classification scenarios, where FLORAL directly improves the robustness of some underlying complex model architectures (such as a pre-trained language model in the case of RoBERTa) on classification tasks.

Overall, with the additional analysis we provided, we extensively analyze FLORAL’s performance across a diverse set of baselines, against a diverse set of label attacks and on different types of datasets. The results highlight consistent and significant performance improvements against state-of-the-art baselines (e.g., in Table 14, approximately 45-50% improvement against RSRNN [2]; in Table 12 up to 17% improvement against RS method [1], etc.).

We are happy to expand on the above points further if further questions remain.

Dear Reviewer bjpg,

As we approach the end of the discussion period, we would like to ask if there is any additional information we can provide that might influence your evaluation of the paper. We believe our previous response with additional experiments conducted has effectively addressed your initial concerns and clarified some points that may have been overlooked.

Thank you once again for your thoughtful consideration of our work.

We thank the reviewer for their feedback. We are glad you found our presentation and theoretical analysis clear and promising. We address your concerns below:

• [Additional experiments] Prompted by your feedback, we have performed additional experiments on the MNIST dataset and reported the results in Appendix-G in the updated manuscript. The results on MNIST are consistent with the other datasets, further showing that FLORAL maintains a higher robust accuracy level when the dataset gets more adversarial. We now show FLORAL’s effectiveness evaluated on diverse datasets from numerical, text, and vision contexts. Furthermore, we considered additional label attacks from literature LFA [1] and alfa-tilt [2] on the Moon and MNIST datasets and evaluated the performance of FLORAL defense against all baselines, as given in Appendix-H-I. Doing so, we now show FLORAL’s performance with extensive experiments on a diverse set of attacks: randomized top-k support vector attack, alfa, lfa, alfa-tilt, and a gradient-based attack. Regarding the size of the datasets, IMDB has 50000 data points including train, validation, and test datasets. IMDB and MNIST are the benchmark datasets used to evaluate the defenses against label poisoning attacks, as also given in the example references provided by the reviewers jdAy and t4Fm. These additional experiments further highlight that FLORAL consistently outperforms the state-of-the-art. More precisely, e.g. in Figure 18, FLORAL defends the best compared to baselines, which also shows its generalizability under different label attack types.

• [Integration with NNs] The direct integration of FLORAL with neural networks, is to use FLORAL as a robust classifier head, as we have shown in the IMDB experiments with RoBERTa + FLORAL integration. That is, extracting the last layer embeddings from an NN model (which will be $x_i$’s in the algorithm), and then training with FLORAL. This way FLORAL helps to learn robust representations. We have already demonstrated in IMDB experiments that FLORAL integrated on top of complex model architecture such as RoBERTa achieves better robustness. The theoretical analysis still holds in this way of integration, as we have no assumptions regarding the inputs. To demonstrate the integration further, we performed additional experiments on NN + FLORAL integration on the Moon and MNIST datasets. As given in Appendix-L in the updated manuscript, with FLORAL integration, NN achieves a higher robust accuracy level, e.g. see Figure 20-(d) that FLORAL provides approximately 10% improvement on the accuracy.

• [Additional dataset] Extensive additional experiments on MNIST presented in Appendix-G, I and H in the updated manuscript further showed that FLORAL achieves higher robust accuracy compared to baseline methods (typically improving by e.g. 4.5% against the next best method in the most adversarial setting (see Figure 18-d)).

• [PGD] We have provided a thorough theoretical analysis of the projected gradient descent algorithm, including its convergence to the global optimum of the proposed training algorithm in Appendix A1-A4.

• Q1: $\mathcal{I}_C$ is the index set for the data points having dual variable value $\geq C$. And, $\mathcal{I}_z$ is the index set for the data points having dual variable value $\in (0,C)$. We apply variable splitting and create these index sets to compute $\mu$ values, which then helps us to quantify the optimal $\lambda^{\star}$ value. We explain this further in the theoretical analysis given in Appendix-A1 (lines 876-877).

• Q2: The Moon dataset is a 2-dimensional synthetic numerical dataset in which each feature takes value in the range $[-2.5,2.5]$. We provide the details (in Section 4) and its illustration in Appendix-C.1.

• • Q2.2: We are unsure about what the reviewer means by “multiple classification tasks”. If multi-class classification is meant, then, our approach is indeed not restricted to binary classification tasks and can be extended to multi-class classification settings using a one-vs-all kernelized SVM strategy. To ensure clarity, we have already provided Algorithm-3 in Appendix-E. Given the straightforward nature of this extension, the details we presented provide a sufficient foundation for applying our approach to multi-class classification.

Overall, with the additional analysis we provided, we extensively analyze FLORAL’s performance across a diverse set of baselines, against a diverse set of label attacks and on different types of datasets. We are happy to expand on the above points further if further questions remain.

[1] Andrea Paudice, Luis Muñoz-González, and Emil C. Lupu. (2019). Label Sanitization Against Label Flipping Poisoning Attacks. In ECML PKDD 2018 Workshop.

[2] Xiao, H., Biggio, B., Nelson, B., Xiao, H., Eckert, C., & Roli, F. (2015). Support vector machines under adversarial label contamination. Neurocomputing, 160, 53-62.

Dear Reviewer 1cTC,

As we approach the end of the discussion period, we would like to ask if there is any additional information we can provide that might influence your evaluation of the paper. We believe our previous response with additional experiments conducted has effectively addressed your initial concerns and clarified some points that may have been overlooked.

Thank you once again for your thoughtful consideration of our work.

Thank you so much for raising your score and the kind words. We are happy to hear that the reviewer appreciates our clarifications. Your feedback has helped us improve our paper, and we sincerely appreciate that.