Revisiting Adversarial Patches for Designing Camera-Agnostic Attacks against Person Detection

Q1: Explanation of hyperparameters and their value ranges for the camera ISP proxy network are needed.

A1: Thank you for your insightful comment regarding the hyperparameter selection for our camera ISP proxy network.

Camera ISPs consist of multiple processing stages. Tseng et al.[1] summarize common imaging pipelines into the following stages: (1) Optics, (2) White Balance & Gain, (3) Demosaicking, (4) Denoising, (5) Color & Tone Correction, and (6) Color Space Conversion & Compression. The first three stages pertain to RAW data acquisition and processing, while the latter three involve operations on RGB values. Since our task primarily focuses on images in RGB format, we chose conditional parameters for the proxy network from the latter three stages. We selected six parameters that significantly impact visual attributes, such as Brightness Contrast Control and Gamma Adjustment, as detailed in Table 2 of the paper. Empirical evaluations demonstrate that these six parameters affect attack performance.

We observed that certain combinations could lead to complete information loss in the image. To balance the quality and diversity of the images generated by the ISP proxy network, we selected the ranges specified in Table 2 of the paper. For parameter $a$, values below 64 may result in images being too dark. For parameter $b$, values below 64 may cause colors to appear washed out. For the $\gamma$ setting, values above the specified range may introduce noise in dark areas, while values below the range can reduce image contrast. For parameter $c$, deviations from the specified interval may lead to poor image quality. For parameter $d$, values below the specified range may result in insufficient spatial filtering, causing excessive noise. For parameter $e$, values less than 1 may not effectively remove noise

We will include a more comprehensive discussion of this topic in the revised manuscript.

[1] Hyperparameter optimization in black-box image processing using differentiable proxies. ACM TOG 2019.

Q2: Essentially, this paper aims to address the issue of camera black-box attacks. However, the authors overlook another black-box scenario, namely the model. The transferability of attack methods across different models is also an important research direction. This paper lacks discussion on the model black-box aspect.

A2: Thank you for your insightful comment. We agree that exploring the transferability of our proposed attack method across different models is an important avenue for future research. We have therefore included additional experiments with other detectors, as shown in the table below.

YOLOv3:

YOLOv8:

The above comparative evaluations were conducted under a black-box setting. The detection models were pretrained on the COCO dataset and then fine-tuned on the INRIAPerson dataset. We observe that, although the effectiveness of our CAP attack decreases compared to white-box attacks (see Table 3 in the paper), it still outperforms other methods.

Your suggestion is valuable and has inspired us to not only consider the black-box camera setting but also the black-box model setting. In future work, we will explore physical adversarial attacks under a double black-box scenario.

Q3: Placing Figure A from the supplementary materials into the main text can help readers better understand the motivation behind this paper.

A3: Thank you for your detailed comment. We agree that including Figure A in the main text would offer a more direct and intuitive understanding of the motivation behind our work. We will carefully consider your suggestion.

Q4: Discussion on the threat model.

A4: Thank you for your careful review. We discuss the threat model from the following three aspects:

• Attack Goal: Our method aims to make the target person evade detection. This is achieved by applying a carefully crafted adversarial perturbation to the target individual's physical appearance, ensuring that the perturbation remains effective across various camera settings.
• Adversary Capability: The adversary has: (1) knowledge of the object detection model and (2) the ability to generate and physically realize adversarial perturbations.
• Design Requirements: (1) The method must be physically realizable and (2) maintain effectiveness across different imaging devices.

We appreciate your high evaluation of our work and the valuable suggestions for future improvements. We believe that the novel perspective introduced by our approach will significantly contribute to the field.

Q1: In the experimental setup, only the patch size is provided, while the input image size is missing.

A1: Thank you for your careful review of our paper. The input image size we used is 640x640, consistent with the official YOLOv5 repository. It is worth noting that YOLOv5 supports both 640x640 and 1280x1280 input image sizes, and we used the former. We will include the specific input image sizes used in our experiments in the revised version of the paper.

Q2: Some key experimental settings are missing. The total number of images used to calculate ASR is not provided.

A2: Thank you for your valuable feedback.

In the digital-space evaluation, we used the publicly available INRIAPerson dataset, which consists of 613 training images and 288 test images. Specifically, the training set contains 3,019 person instances, while the test set contains 855 person instances. Therefore, the ASR in the digital space is calculated based on 288 images containing 855 person instances.

In the physical-space evaluation, we collected images using 6 cameras at 4 different times to avoid interference from unrelated factors. For each patch setting, 5 images were captured per camera per session, resulting in 6x4x5=120 images per patch. We compared 6 adversarial patches in the physical domain, yielding a total of 6x120=720 images. The ASR in the physical space is thus calculated based on these 720 images.

We will clarify the specific number of images in the revised version of the paper.

Q3: The robustness of the proposed CAP against real-world disturbances (e.g., random noise, blur, etc.) should also be discussed.

A3: Thank you for your insightful comment. We acknowledge the importance of robustness in real-world disturbances. During the training of CAP, we implemented techniques to enhance robustness, specifically (1) adding random noise and (2) adding random rotations. These measures strengthen the robustness of the CAP attack, supporting its effectiveness in real-world scenarios, as demonstrated in Section 4.3 of the paper.

Blur, commonly caused by camera motion in videos, was not considered in our evaluation as the detectors we targeted, such as the YOLO series and Faster R-CNN, are image-based. Your comment highlights an important area for future research, particularly the impact of blur on physical adversarial attacks. To the best of our knowledge, this has not been explored extensively. In future work, we plan to investigate physical adversarial attacks on video detection models and evaluate the effects of blur. We believe that modeling video motion and incorporating temporal frame consistency during training are promising approaches.

We appreciate your positive feedback on our work and the valuable suggestions for our future research. We believe that the new perspective offered by our method will significantly contribute to the field.

Q1: While the authors experimental evaluation is quite comprehensive [...].

A1: Thank you for your valuable feedback. We acknowledge that evaluating the performance of our approach on other detectors would be meaningful. To address your concerns, we have conducted additional experiments, as detailed in the table below.

YOLOv3:

YOLOv8:

The above comparative evaluations were conducted under a black-box setting. The detection models were pretrained on the COCO dataset and then fine-tuned on the INRIAPerson dataset. We observe that, although the effectiveness of our CAP attack decreases compared to white-box attacks (see Table 3 in the paper), it still outperforms other methods.

Your suggestion is valuable and has inspired us to not only consider the black-box camera setting but also the black-box model setting. In future work, we will explore physical adversarial attacks under a double black-box scenario.

Q2: Video artifacts.

A2: Thank you for your insightful comment. The difference between video capture and image capture indeed affects attack effectiveness. The key distinction lies in the inherent camera and scene motion in video, which leads to scene shifts and introduces motion blur-related artifacts that significantly impact attack effectiveness. We observed that attacks sometimes failed in the presence of blur.

Our CAP attack focuses on images because detectors like the YOLO series and Faster R-CNN are image-based. Consequently, we did not consider the effects of video artifacts. However, to enhance the generalization of the adversarial perturbations, we incorporated tricks such as (1) adding random noise and (2) applying random rotations during the optimization process.

Your comment provides valuable insights for future research, especially regarding physical adversarial attacks on video detection models. To the best of our knowledge, no work has been done in this area. The generalization from images to videos remains an open problem. We believe that modeling video motion and considering temporal frame consistency during training are promising approaches.

Q3: Camer number.

A3: Thank you for your valuable comment. We chose to design and validate our method on smartphone cameras (iPhone, Redmi, Huawei, and Samsung) and typical consumer cameras (Sony and Canon) first because these cameras are portable, widely used, and more susceptible to attacks. In contrast, they are less reliable than industrial cameras, especially in challenging environments like high temperatures, humidity, and electromagnetic interference.

We agree that expanding the range of cameras would further enhance the applicability of our method. In future work, we plan to extend our evaluation by using industrial cameras to validate the attack's effectiveness. The following table lists industrial cameras commonly used in autonomous driving and video surveillance that we have surveyed.

Q4: Negative societal impact.

A4: We fully acknowledge it is crucial to consider these potential negative societal impacts. The ethical statement about our work is given below:

Our work successfully achieves physical adversarial attacks in person detection tasks. Given the effectiveness of our attack method across various imaging devices, its real-world application is feasible. This exposes potential security risks in existing DNNs-based applications, particularly when the technology is leveraged for malicious purposes. We advocate for the responsible and ethical use of technology. Furthermore, we offer comprehensive methodological descriptions and openly address the implications of our work, encouraging discourse within and beyond the scientific community to contribute to the advancement of trustworthy and dependable AI.

Additionally, to mitigate the potential risks of our CAP attacks, we discuss defense strategies in Section 4.5 of the paper. Through evaluation across three defense strategies, the experimental results indicate that adversarial training effectively mitigates our CAP attack in both digital and physical spaces. Moving forward, we will actively collaborate with the security community to explore the broader implications of our work and identify further mitigation strategies.

Q5: Random seed.

A5: Thank you for your careful review. As per your suggestion, we will remove the statement about the random seed from the main paper and include it in the appendix, leaving more space in the main text for more valuable discussions.

Q6: There is duplicated text on lines 254-255.

A6: Thank you for bringing this to our attention. We will eliminate the redundancy and thoroughly proofread the entire paper to ensure clarity throughout.

We appreciate your high praise for our work and the valuable suggestions for our future research. We believe that the new perspective offered by our method will significantly contribute to the field.

Q1: Typographical errors [...].

A1: We have made the following corrections:

• Corrected the citation in line 52: Zhang et al.[40] employed [...].
• Corrected the citation in line 129: as Zhang et al.[40] demonstrated [...].
• Removed the duplication in line 255.

We will thoroughly proofread the paper.

Q2: Flowchart.

A2: Thanks for your feedback. The revised flowchart can be found in the one-page PDF file.

Q3: Although different smartphone [...].

A3: Thanks for your comment. Smartphone and consumer cameras are portable and widely used but less reliable in harsh environments (e.g., high temperatures, electromagnetic interference) compared to industrial cameras used in autonomous driving and surveillance. Exploring their security implications is crucial, so we initially validated our method on these cameras.

To substantiate the claim in the paper, our approach introduces a wide parameter space for the ISP proxy network and uses six different cameras, offering a broad sampling space for training and evaluating.

In future work, we plan to include industrial cameras to enhance applicability in autonomous driving and surveillance. The following table lists commonly used industrial cameras we have surveyed.

Q4: Theoretical analysis.

A4: As outlined in Sections 3.2 and 3.4, our approach is inspired by the GAN theoretical framework. Attacker acts as Generator, producing adversarial perturbations, while Defender acts as Discriminator, optimizing hyperparams. Our alternating optimization mirrors GAN convergence analysis. We provide detailed theoretical proof:

Global Optimality of the Adversarial Optimization Framework

We begin by analyzing the optimal Defender strategy given any Attacker strategy.

Proposition 1: For fixed Attacker strategy, optimal Defender strategy minimizes adversarial perturbation effectiveness.

Proof: Attacker aims to maximize error induced by adversarial perturbations, while Defender aims to minimize this error. Formally, let $P$ represent the perturbations and $\theta_d$ the hyperparams. Attacker's objective:

$$L_A = \max _P \mathbb{E} _{x \sim p _\text{data}} \left[ \ell(f(x + P), y) \right]$$

where $\ell$ is the loss function, $f$ is the neural network, and $y$ are the GT labels.

The Defender seeks to minimize the effectiveness of these perturbations by optimizing $\theta_d$:

$$L_D = \min _{\theta_d} \mathbb{E} _{x \sim p _\text{data}} \left[ \ell(f(g(x + P; \theta_d)), y) \right]$$

where $g$ represents the ISP proxy network processing.

The optimal Defender strategy $\theta_d^*$ satisfies:

$$\theta_d^* = \arg \min_{\theta_d} L_D$$

Given this optimal strategy, the Attacker's objective function becomes:

$$L_A^* = \max_P L_D(\theta_d^*)$$

The interplay between the Attacker and Defender can be modeled as a minimax game:

$$\min _{\theta _d} \max _P \mathbb{E} _{x \sim p _\text{data}} \left[ \ell(f(g(x + P; \theta_d)), y) \right]$$

Since the ISP proxy network aims to minimize the impact of adversarial perturbations, the optimal strategy for the Defender minimizes the Attacker's maximum achievable loss, concluding the proof.

Convergence of the Optimization Algorithm

Proposition 2: If Attacker and Defender have sufficient capacity, and at each iteration of optimization algorithm, Attacker and Defender optimize respective objectives, adversarial perturbations converge to equilibrium where effectiveness is minimized by ISP proxy network.

Proof: The optimization algorithm follows an alternating training strategy, akin to the GAN. At each step, Attacker optimizes the perturbations $P$ while keeping the ISP parameters $\theta_d$ fixed, and vice versa.

Attacker’s update rule is:

$$P^{(t+1)} = P^{(t)} + \alpha \nabla_P L_A$$

Defender’s update rule is:

$$\theta_d^{(t+1)} = \theta_d^{(t)} - \beta \nabla_{\theta_d} L_D$$

By iteratively updating $P$ and $\theta_d$ in this manner, the algorithm performs gradient descent on a convex-concave function, ensuring convergence to a local Nash equilibrium. At this equilibrium, the adversarial perturbations $P^*$ are countered effectively by the optimized ISP parameters $\theta_d^*$, concluding the proof.

Q5: Differences of the patch training process?

A5: Mainstream patch-based methods apply a random patch to the target image and update it to maximize the discrepancy between predictions and GT, analogous to the Attacker in our method. Our approach further introduces a Defender, represented by a ISP proxy network. This addition enables the Attacker and Defender to form an adversarial optimization framework, enhancing the attack transferability across different cameras.

Q6: Targeted or untargeted attacks.

A6: Our approach is an untargeted attack. In classification tasks, targeted attacks mislead the classifier into categorizing the target instance as a specific incorrect class, while untargeted attacks cause the classifier to misclassify the target instance into any incorrect class. Based on this criterion, we tend to classify the task of concealing person instances as an untargeted attack.

Our method, as an untargeted attack, was evaluated using AP. However, AP alone can be misleading, especially in multi-box detection scenarios where AP may be low despite ineffective attacks (see Section B of our supplementary material). ASR compensates for this limitation of AP, as it is not affected by an increase in True Positive samples. Thus, we employ both AP and ASR.

Lastly, we would like to thank you for acknowledging the strengths of our method, as you noted: (1) a critical [...] and (2) This novel [...]. We hope our responses address your concerns and contribute to an improved rating for our paper.