$DPOT_{L_0}$: Concealing Backdoored model updates in Federated Learning by Data Poisoning with $L_0$-norm-bounded Optimized Triggers

1. I am confused about the authors' threat model. I do not understand why a malicious client cannot manipulate their local training process (Lines 128-133).

We found this Wikipedia page very helpful for understanding our threat model: https://en.wikipedia.org/wiki/Trusted_execution_environment. It introduces the concept of TEEs and lists many commercial TEEs. Our threat model assumes that the local training process is executed within TEEs, which the FL server can verify by having each TEE attest to its software and inputs. TEEs greatly increase the difficulty of malicious manipulation.

2. Since the triggers are added to the input data rather than the features, Proposition 5.1 cannot fully demonstrate the effectiveness of the authors' method.

Following the problem setup of the feature learning theory paper (line 589), we consider a datum is composed by various patterns with some of patterns are critical for the classification, which we model as linear vectors and name them as features. Backdoor data's features are trigger patterns. We used $K$ to represent a dataset containing only benign features, and $K\cup K_{adv}$ to represent a dataset containing both benign and backdoor features. Proposition 5.1 correctly explains theoretical insights of our method in dataset-level, rather than data-level.

3. In federated learning, not every client is selected to participate in each round (Line 269).

We experimented with scenarios where not every client is selected to participate in each round by randomly sampling a portion of clients for each round training, and used Selection Ratio (SR) to determine this portion. We present results for SR = 0.5 and SR = 1, comparing $DPOT_{L_0}$ with FT and DFT against three different defense strategies on CIFAR-10 as the main training task. As shown in the following table, $DPOT_{L_0}$ demonstrates better attack effectiveness in both SR settings and achieves a sufficient ASR faster than the other attacks.

4. Although the authors present a new backdoor attack method, I currently do not see its practical value. In my view, it merely utilizes gradient information to determine the location of the triggers.

$DPOT_{L_0}$ considers a FL settings where all clients are run in a TEE environment, which has strict limitation over adversaries' capability in comparison to previous works. This highlights the enhanced practical value of $DPOT_{L_0}$.

5. Some federated backdoor attacks [1,2,3] also employ dynamic triggers.

[1] Iba is one of the baselines that we did thorough comparison with (line 281, 311). Please see section 6.6 for our comparison results from 4 different dimensions. We will respectively mention reference [2], [3] for their contribution of employing dynamic triggers against Personalized Federated Learning in the related work, and thank you for sharing these references with us.

6. The writing needs improvement. At present, the motivation behind the authors' method is limited.

We'd love to adopt suggestions to improve our writing. The motivation behind $DPOT_{L_0}$ is to expose the vulnerabilities in current FL defenses and emphasize the need for more robust countermeasures by demonstrating the effectiveness of the introduced data-poisoning-only backdoor attacks.

7. Since the authors used a Non-IID setting, they could test the proposed method's attack performance for personalized federated learning.

Our work focuses on studying the security performance of the fundamental FL structure introduced by McMahan et al. (2017), with selected attack and defense baselines all served for this structure. We agree that exploring the attack performance of $DPOT_{L_0}$ on advanced FL structures, such as Personalized Federated Learning, Vertical Federated Learning, and Federated Transfer Learning, would be a highly promising direction for future work.

1. As can be seen from Figure 2, the generated trigger is distinguishable by the human eye. Does this lead to smart clients using simple data filtering/clear methods to suppress backdoor attacks? (Simple defenses against poisoning data being filtered and deleted need to be discussed to reveal the actual value of the proposed methods)

We agree that our generated triggers are distinguishable by the human eyes.

During the FL training, malicious clients as the attack executors should not want to filter triggers out, and benign clients without having trigger information are unable to filter triggers out. Even though we let TEE carry out clear methods on data, backdoor data still cannot transform to benign data since their labels have been changed. An adaptive change on attack can be constraining both $L_\infty$ and $L_0$ bounds of trigger during optimization so that the magnitude of trigger pixels can be better controlled.

During the inference stage, smart users can clear the images before inputing them into the victim FL model to bypass backdoor attacks, but this clear method might also alter features in benign images which will degrade main-task accuracy. Nonetheless, we acknowledge that improving data cleansing methods to more accurately filter backdoor triggers is a valuable defense strategy worth exploring.

2. What insights does the theoretical analysis provided in section 5 specifically provide for the design of backdoor attacks?

Thank you for this valuable feedback. Section 5 provided the theoretical insights that the difference in update directions between benign and malicious objectives is bounded by the error of the malicious data on the model. Section 4 introduced how $DPOT_{L_0}$ decreases backdoor data's error (loss) on the global model, and section 5 provided justification that the error reduction can help conceal malicious model updates among benign ones. We will add the following formula to paper for better connecting section 4 and section 5.

$\min \epsilon_{adv} = \min_{\tau} \quad \frac{1}{|D|} \sum_{x \in D} \mathit{Loss}(W_g(x \odot \tau), y_t)$

By optimizing $L_0$-norm bounded trigger $\tau$ to minimize the error $\epsilon_{adv}$, $DPOT_{L_0}$ not only reduces backdoor loss but also conceals malicious model updates (proposition 5.1).

1. What is the direct relationship between Section 5 and Section 4? There is no clear explanation provided to demonstrate how the design of DPOT is benefited from the theoretical analysis. I would appreciate a brief explanation here to emphasize how Section 5 can help the reader better understand DPOT.

Thank you for this valuable feedback and advice. Section 5 provided the theoretical insights that the difference in update directions between benign and malicious objectives is bounded by the error of the malicious data on the model. Section 4 introduced how $DPOT_{L_0}$ decreases backdoor data's error (loss) on the global model, and section 5 provided justification that the error reduction can help conceal malicious model updates among benign ones. We will add the following formula to paper for better connecting section 4 and section 5.

$\min \epsilon_{adv} = \min_{\tau} \quad \frac{1}{|D|} \sum_{x \in D} \mathit{Loss}(W_g(x \odot \tau), y_t)$

By optimizing $L_0$-norm bounded trigger $\tau$ to minimize the error $\epsilon_{adv}$, $DPOT_{L_0}$ not only reduces backdoor loss but also conceals malicious model updates (proposition 5.1).

2. Section 4.2 is confusing. If the trigger size in the following part is simply pre-defined, it is unnecessary to have this section here. Besides, how the trigger size is chosen for the evaluation?

Thank you for this feedback. Section 4.2 introduces the strategy we use to pre-define the trigger size. We determined the trigger size by ensuring that the accuracy drop of poisoned data, predicted as benign by an un-attacked model, does not exceed 30% (subtlety goal). Concrete examples of trigger size selection used in evaluation are provided in Appendix J due to space limitations, as referenced in line 418. Based on your valuable feedback, we will improve the clarity of our statement and mention those examples earlier in the method section for better readability.

We found an experiment suggested by Reviewer Vb6j very interesting, and would like to share it here.

3. Duration of attack effectiveness

We compared $DPOT_{L_0}$ and $FT$ on the duration of their effectiveness after attack termination. Using FedAvg on the CIFAR-10 dataset, we first applied 50 rounds of data poisoning for both attacks reached 100% ASR. We then stopped the poisoning and recorded the training rounds needed for ASR to drop to 50%.

Testing with different learning rates (lr), we found that higher lr shortened the attack's duration. As shown in the table, $DPOT_{L_0}$ consistently had a longer duration than $FT$ across all lr. A possible explanation is that $DPOT_{L_0}$ disperses trigger pixels across the image, activating more neurons and reinforcing trigger feature retention. In contrast, $FT$ clusters trigger pixels in a corner, affecting fewer neurons and leading to faster forgetting.

Thank you for the effort you put into providing feedback and advice!

1. Limited studies of MCR and Data poison rate

Due to the space limitation, we discussed ablation studies of MCR and Data poison rate in Appendix L and N. We also discussed ablation studies of Trigger size and Non-iid degree in Appendix M and O.

2. Potential countermeasures against $DPOT_{L_0}$

The real issue exposed by crafty data-poisoning attacks like $DPOT_{L_0}$ is a private data governance problem—how to supervise clients' private data to ensure alignment with the common goal of the entire FL system. Addressing this would need a security framework to protect the entire FL lifespan—from clients deciding to form an FL group to the completion of their FL training. Ensuring both privacy and security may demand system architecture or infrastructure-level interventions beyond ML solutions. We hope to see such frameworks developed collaboratively by the ML and system communities to make FL more practical in real-world applications.

3. Connection between $L_0$ attacks and backdoor attacks

Can we understand the $L_0$ attacks as $L_0$-norm-bounded adversarial examples used in testing? We cited Papernot et al. (2016) (line 83) for reference. The similarity of $L_0$ adversarial example and $L_0$ backdoor example is that they both constrain the number of altered pixels. The difference is that adversarial examples vary pixel changes per image, while backdoor examples use consistent patterns (shape, value, and placement) to embed a learnable backdoor feature. Unlike adversarial examples, backdoor examples should avoid introducing excessive new features to prevent hindering the main task's convergence.

4. Empirically validate the bound in the non-linear case

We generated 20 random data points (each has 50 dimensions) to simulate $K$ and 20 random target values $y$ for them. $K_{adv}$ contains 5 random points with specified targets $y_{adv}$ and $\epsilon_{adv} = w(K_{adv}) - y_{adv}$, where $w$ is a model with random parameters. We defined $\Delta w_K = \frac{\partial \Vert w(K) - y\Vert_2}{\partial w}$ and $\Delta w_{K \cup K_{adv}} = \frac{\partial\Vert w([K, K_{adv}]) - [y, y_{adv}]\Vert_2}{\partial w}$, and studied the relationship between $\Vert\epsilon_{adv}\Vert_2$ and their distance $\Vert\Delta w_K - \Delta w_{K \cup K_{adv}}\Vert_2$.

We tested three models: Linear, Two-layer (ReLU activation between linear layers), and LeNet5 (2 convolutional and 3 linear layers). More details are provided here.

Varying $\Vert\epsilon_{adv}\Vert_2$ by changing $y_{adv}$, we recorded the corresponding gradient distance changing along with it. The results for different model architectures are presented in this figure, all of which align with Proposition 5.1. For LeNet5, we also demonstrated the gradient distances separately for its convolutional layers and linear layers in this figure, finding both bounded by $\Vert\epsilon_{adv}\Vert_2$ with different coefficients. These results indicate that Proposition 5.1 can be applied to non-linear neural networks.

5. Trigger training data from different distributions

To study this insightful proposal, we constructed Trigger Training Datasets (TTDs) using Out-Of-Distribution (OOD) data and evaluated the impact on $DPOT_{L_0}$'s performance. For the Fashion MNIST task, we used MNIST data to train triggers, and for CIFAR10, we selected data from CIFAR100 that are in different categories to CIFAR10. Triggers generated by OOD data can be found here, and the results comparing In-Distribution (ID) and OOD TTDs are shown here.

Triggers generated from OOD data might not be the optimal ones to apply to ID victim data, therefore we saw OOD TTDs producing less effective results compared to ID TTDs. Nonetheless, they still show strong attack performance, making this a promising direction for further exploration.

6. Attack in non-consecutive rounds

In our main evaluation, a few pre-selected attackers participate in every FL round. We also tested non-consecutive attacks, discussed in Appendix P, and found that non-consecutive attacks can enhance $DPOT_{L_0}$'s effectiveness against certain defenses.

7. The number of rounds needed to achieve the attack goal

We set $ASR$ = 50% as the attack goal and recorded the number of rounds required for different attacks to achieve it under various training tasks and defenses. The results can be found via this Link. $DPOT_{L_0}$ shows the fastest speed in reaching the attack goal among all attacks.

8. Duration of attack effectiveness

Please see our response to Reviewer KkLM, section 3.