Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detection

The author would like to thank the reviewer for appreciating our clear problem formulation and the effectiveness of our proposed attack. We provide more evaluation detail, conduct additional ablation, and explain the motivation behind our method, which we hope can address your concerns.

Q1: In Eq. (2), as stated, the blending transformation needs two real samples, however, the variable in function T^b only has one sample instead of a pair of real samples. For example, in Face X-ray, the fake face image is generated via two different real ones. So I think the formulation should be adjusted to make it more reasonable.

A1: Thanks for your suggestions. Our current formulation is based on the SBI method where the two real samples are from the same source, which can be considered as a special case. We appreciate your feedback and we have revised the formulation to a more general form that can represent different blending artifact methods, including SBI and Face X-ray, etc.

Q2: In the section on ‘Stealthiness of Backdoor Attacks’, Table 2 shows the qualitative results among different backdoor attack methods. However, the paper does not mention the number of samples for evaluation.

A2: Thanks for your feedback. In the 'Stealthiness of Backdoor Attacks' section, we conducted a quantitative (PSNR and L_inf) evaluation using the fake subset of the FF++ dataset's test set. This subset consists of 560 fake videos, and we extracted 32 frames per video, resulting in a total of 17,920 samples. For the human perception studies, we randomly selected 5 images from the aforementioned subset and applied all 6 attacks, resulting in a total of 30 samples per participant. We have included these details in the revision.

Q3: The influence of the hyperparameter ‘a’ is ambiguous. Moreover, the comparison with/without ‘alpha’ is missing and hence makes readers confused about the effectiveness brought by the proposed relative embedding. It is better to show the qualitative results and the quantitative backdoor attack performance.

A3: Thanks for your feedback. We would like to address your concerns as follows:

1. The hyperparameter 'a' serves as a scaler to to determine the maximum magnitude of the backdoor trigger. A higher 'a' value results in a less stealthy trigger, while a lower 'a' value enhances stealthiness. In our experiment, we set 'a' to a low value (0.05) to prioritize stealthiness of the trigger.
2. The hyperparameter 'alpha' is defined as 'alpha' = a * x_k / 255, where 'x_k' represents the pixel values of the image. When embedding the trigger into the image, we adjust the pixel values of the trigger based on the magnitude of the pixel values in the image. This adjustment further enhances the stealthiness of the trigger.
3. We clarify that training without 'alpha' or setting 'alpha' to zero is equivalent to benign training without any backdoor attack. In our paper, we have presented the results of benign training in the 'w/o attack' line of Table 1. Furthermore, we are happy to follow your suggestions and conduct an extra ablation study to investigate the impact of the hyperparameter 'alpha'. This involves varying the blending type (relative or absolute) and the blending ratio (value of 'a') in the table mentioned. In the table, the 'relative' line represents the method we used in our paper, where 'alpha' is defined as 'alpha' = a * x_k / 255. On the other hand, the 'absolute' line indicates the experiment we conducted as a reference, where 'alpha' is defined as 'alpha' = a.
4. From the results, we can draw observations as follows:
   1. As the value of 'a' increases, we can achieve higher attack performance but lower PSNR. This is because the tradeoff between the attack performance and the stealthiness of the backdoor trigger.
   2. Although absolute embedding yields better attack performance, this improvement comes at the cost of lower PSNR.

I wish to convey my appreciation for your considerate responses and meticulous revisions. However, some of the papers related to face forgery detection should be discussed in the related work as previous detection methods is relatively old:

[1] Shao R, Wu T, Liu Z. Detecting and recovering sequential deepfake manipulation[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 712-728.

[2] Xia R, Liu D, Li J, et al. MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection[J]. arXiv preprint arXiv:2307.02733, 2023.

[3] Shao R, Wu T, Liu Z. Robust Sequential DeepFake Detection[J]. arXiv preprint arXiv:2309.14991, 2023.

Q4: In the section on trigger generation, the authors should provide a comprehensive description of the generator's overall loss function and its basic architecture.

A4: Thank you for your suggestion. We would like to provide a more detailed description of the generator's basic architecture and the overall loss function:

1. Basic Architecture of the Generator (G):

• 1x1, conv, 512, stride=1, padding=3, BN
• 4x4, deconv, 256, stride=2, padding=3, BN
• 4x4, deconv, 256, stride=2, padding=3, BN
• 4x4, deconv, 256, stride=2, padding=3, BN
• 4x4, deconv, 256, stride=2, padding=3, BN
• 4x4, deconv, 128, stride=2, padding=3, BN
• 4x4, deconv, 3, stride=2, padding=3, BN

In the above description, "KxK, conv/deconv, C, stride=S, padding=P, BN" indicates a convolutional or deconvolutional layer with a KxK kernel, C output filters, stride S, and padding P, followed by batch normalization (BN).

2. Overall Loss Function for the Generator: The overall loss function for the generator G(z) can be found in Equation 10 of the paper. It is represented as L_g = − log ||K(v)⊗G(z)||_1. Here, K(v) denotes our proposed convolutional kernel as described in Equation 8, and G(z) represents the output of the generator with a latent variable z ~ N(0,1) as input. We train the generator G(z) with L_g as the loss function.

The author would like to thank the reviewer for appreciating our motivation and high steathiness and performance of our proposed attack. We provide detailed explanation of mechanism behind kernel size and further evaluate existing defense and discuss potential defense strategies, which we hope can address your concerns.

Q1: The paper could provide a discussion on the limitations of the proposed method, ideally placed in the appendix.

A1: Thanks for your suggestions. We would like to discuss the limitations of the proposed method as follows:

1. The proposed methods are not optimal. While our methods focus on the common transformations in face forgery and generally demonstrate effectiveness across different methods, they are not the optimal for specific methods.
2. The attack performance is partially dependent on the size of the kernel used for trigger generation. The choice of kernel size can have an impact on the effectiveness of the attack.

Finally, we have included this discussion to the appendix.

Q2: The choice of different kernel sizes for the trigger can impact the attack performance on various forgery detection models. The rationale behind such an impact remains unclear. Additionally, the authors' decision to finalize a size of 5 for the kernel lacks a clear justification.

A2: Thanks for your suggestions. We would address your concern as follows:

1. As mentioned in Section 5.3 of the paper, the choice of kernel size indicates the range of translation that we aim to optimize. This range is based on the attacker's assumption on the translation amplitude during the synthesis of fake data. A larger kernel size suggests that the triggers are optimized over a broader translation range. However, a larger kernel size may not always lead to better performance. This is because the triggers could be optimized to adapt to a larger range of translations, which may not be the best match for the translation amplitude during the synthesis of fake data.
2. Regarding the decision to use a relatively small kernel size of 5, this is based on the consideration that the distinction in facial boundaries of the synthesized fake images can be really small, in order to make the synthesized fake images appear more natural and less suspicious to the human eye.

Q3: Table 4 indicates that existing defense mechanisms fail against the proposed attack. It remains ambiguous whether this failure is specific to the attack introduced by the authors or if it's a general shortcoming for other attacks as well. Furthermore, it would be valuable if the authors could analyze existing attacks to suggest potential defense strategies.

A3: Thanks for your suggestions. Firstly, we conduct additional experiments to evaluate the performance of existing defense methods against the current attack of best performance, specifically the blended attack. These experiments are conducted on the SBI detection method on FF++ dataset, and the results are presented in the following table. From the results, we observe that under the clean label setting, the four defense methods tested also fail to effectively defend against the existing attack in face forgery detection. This finding suggests that while attacking blending artifact methods is more challenging, once a successful attack is launched, it establishes a strong backdoor connection that existing defense mechanisms may struggle to mitigate.

Regarding the potential defense strategies, we would like to offer the following insights:

1. During our fine-pruning defense experiments, we observed that the neural activation in the linear layer is sparse. For example, we selectively prune a significant portion (e.g., 99%) of the less-contributing neurons and fine-tune the model on clean data. Even with such aggressive pruning, the model is able to maintain performance. This result inspires a potential future direction in which we could explore more fine-grained pruning techniques on the remaining 1% of neurons.
2. In the clean label setting, where only the images are modified while the labels remain unchanged, we hypothesize that adding noise to the images during training can help destroy the trigger pattern and mitigate the establishment of the backdoor connection. This approach may be particularly effective when the magnitude of the backdoor trigger is kept small in order to ensure stealthiness.

Q4: The reported performance of Face X-ray is lower than that in the original paper and in the SBI paper. I am wondering if there is something wrong with the experiments.

A4: Thank you for bringing these points to our attention. The performance difference on clean samples can be attributed to two main factors:

1. Data version: The FF++ datasets have multiple versions, including the raw (original) version and compressed versions such as c23(HQ) and c40(LQ). As mentioned in Section 4.4 of the SBI paper [1], the original performance of both Face X-ray and SBI models was evaluated on the raw version of the data. But in our experiments, we used the compressed (c23) version of the data for both training and testing. It is known that the performance on clean compressed data may experience a drop [2]. On the other hand, the CDF dataset has only one version, and our reported performance on SBI is 93.10, which is close to the results reported in the original paper (93.18) and the official GitHub repository (92.87%).
2. Unified face extraction process: To avoid possible bias in the data preprocessing, we employed a unified face extraction process proposed by SBI, which includes consistent parameters such as the number of frames per video and the area of the extracted faces, for all three face forgery detection methods. This standardized approach may lead to performance differences compared to original paper.

In addition, we conducted evaluations on the raw version of FF++ dataset. The results are presented in the following table.

1. We evaluate the clean performance on the raw data, which aligns with the evaluation approach used in the original papers. The 'original w/o attack' line in the table represents the results reported in the original paper. The results indicate that our performance on clean raw data is close to the original reported results.
2. We also evaluate the attack performance on the raw data. We compare the existing attack with the best performance with our method on the raw data. We observe that the attack performance, as indicated by BD-AUC, is higher on the raw data compared to the compressed (c23) data. According to these results, our method still demonstrates superior performance compared to the existing attack on the raw data.

The author would like to thank the reviewer for appreciating our comprehensive benchmark and our proposed attack that is effective across different face forgery methods. We add additional benchmark on frequency-based attack and evaluate the robustness against various image preprocessing algorithms, which we hope can address your concerns.

Q1: This paper is not the first in the literature to address backdoor attacks in face forgery detection. Cao et al. [A] have already investigated this problem. Therefore, the first contribution is invalid.

A1: Thank you for your correction. We have thoroughly reviewed the mentioned paper by Cao et al. [A] and acknowledge its contribution to backdoor attacks in face forgery detection. We apologize for any confusion caused by our previous statement regarding the "first" contribution. Additionally, we would like to highlight the distinctions and emphasis of our paper in comparison to the mentioned work:

1. Scope of investigation: While the mentioned paper focuses on investigating a single backdoor attack, specifically Badnet, in face forgery detection, our paper goes beyond and provides a more comprehensive analysis. We benchmark five different backdoor attacks and propose a novel attack method in this field.
2. Extension to blending artifact detection methods: Our research also extends to the examination of backdoor attacks on blending artifact detection methods, specifically SBI and Face X-ray. These methods are more challenging to attack compared to the previously studied detection methods.

Finally, we have made the necessary modifications and added appropriate citations in the revision.

Q2: The use of "b" in the equations in section 3 is confusing. b presents both "blending" and "remaining clean."

A2: Thank you for pointing out the issue. In the revision, we clarify and differentiate the symbols used to represent "blending" and "remaining clean". Specifically, we use "b" to represent "blending" and introduce 'c' to represent "remaining clean".

Q3: The benchmark should include some frequency-based backdoor attacks like [B].

A3: We appreciate your suggestion and conduct experiments that involve the mentioned frequency-based attack, FTrojan [B]. The results are presented in the following table. Upon analysis, our attack outperforms the frequency-based attack, FTrojan [B], on three face forgery detection models.