The Fluorescent Veil: A Stealthy and Effective Physical Adversarial Patch Against Traffic Sign Recognition

We sincerely thank you for your insightful and positive feedback!

Q-1. UV light is invisible, using a UV lamp may be visible. How can the attacker ensure that the UV lamp remains covert? Moreover, in some scenarios, such as highways, where traffic signs are usually placed high, can the attack still be successful at this point? It would be helpful if this aspect could be explained to alleviate the concerns about the stealthiness.

A-1. We tested the impact of distance between UV lamp and traffic sign in Appendix Section D and found that there is still an attack success rate of at least 81% at a distance of 8 meters. This distance is sufficient for an attacker to place the UV lamp in the bushes near the traffic sign. In addition, since the UV light is invisible to the naked eye, the victim cannot locate the light source, and the attack device is in the bushes, further increasing the stealthiness of the attack. For elevated traffic signs commonly found on highways, attackers can leverage drones to carry out the attack. By equipping a drone with a UV lamp and hovering it at a specific position, the attacker can activate the perturbation without physical access to the sign. The drone only needs to remain in position for a few seconds during the attack and can quickly leave the area afterward, making the attack highly stealthy and flexible.

Q-2. The paper should further discuss the feasibility of defenses, e.g., could a UV filter defend against the attacks in this paper? Relying solely on high-definition maps may not be sufficient, especially in cases where traffic signs have been replaced by workers but the maps are not yet updated.

A-2. A UV filter cannot defend against FIPatch. First, they cannot stop fluorescent materials from absorbing UV light, and the emitted visible light is still captured by cameras. Second, UV filters hide the attack source by blocking UV light in images. Apart from high-definition maps, we believe that collaborative perception among multiple vehicles is also a promising defense strategy. Specifically, while an attacker may launch adversarial attacks using fluorescent ink against a specific vehicle, other unaffected vehicles are still able to recognize the traffic sign correctly. By leveraging recognition results from nearby vehicles on the same road, the victim’s vehicle can correct its prediction. This cross-vehicle information sharing mechanism can significantly enhance the system’s robustness against adversarial attacks.

Q-3. Although physical experiments are thorough, the impact is only measured at the AI component level (classification/detection), not on full AV systems (e.g., driving decisions).

A-3. Our paper has mentioned this issue in the limitation. For fairness, we followed prior studies [1]-(USENIX-2023), [2]-(NDSS-2022), [3]-(USENIX-2021), and mentioned the AD systems in limitation. If insist, we could provide results on real-AD systems.

Reference

[1] Wenjun Zhu, Xiaoyu Ji, Yushi Cheng, Shibo Zhang, and Wenyuan Xu. Tpatch: A triggered physical adversarial patch. In USENIX Security, 2023.

[2] Wei Jia, Zhaojun Lu, Haichun Zhang, Zhenglin Liu, Jie Wang, and Gang Qu. Fooling the eyes of autonomous vehicles: Robust physical adversarial examples against traffic sign recognition systems. NDSS, 2022.

[3] Giulio Lovisotto, Henry Turner, Ivo Sluganovic, Martin Strohmeier, and Ivan Martinovic. SLAP: Improving physical adversarial examples with Short-Lived adversarial perturbations. In USENIX Security, 2021.

Q-1. Lacks new methodological contributions.

A-1. We highlight again that we explore a new adversarial attack strategy by using the fluorescent ink to achieve a more stealthy and effective attack in the physical world. Compared with existing methods, our FIPatch offers the following advantages:

1. High effectiveness: Existing methods are often vulnerable to defense strategies, such as optical filters or physical signal protection mechanisms. In contrast, our approach cannot be easily defended against by current techniques and demonstrates strong robustness in the physical world. This is validated by both digital and physical experiments.

2. Unconspicuous: Our method supports flexible triggering. It remains invisible when inactive and minimizes the perturbation area when activated. In the physical world, the perturbation only needs to be active for a short time. When a self-driving car approaches the stop sign, even if it fails to recognize the stop sign for merely a short time window, it can lead to a fatal accident.

Although fluorescent ink enables invisible and actively triggered features, designing an effective and robust physical adversarial patch introduces new challenges.

(1) It is challenging to simulate fluorescent effects and determine the optimal choices of massive factors in fluorescent ink, e.g., color, transparency, and size, for achieving high attack effectiveness.

(2) Fluorescent effects are easily influenced by real-world environments, including surroundings, ambient light, and vehicle distance.

Existing methods do not address these challenges, and most require manual annotation of traffic signs. To address the challenge (1), we first develop a color-edge fusion method to automatically locate traffic signs, enabling precise application of fluorescent ink to the signs, rather than invalid backgrounds. Second, to effectively simulate fluorescent effects, we model fluorescent perturbations on traffic signs by defining the various critical parameters of fluorescent ink, including colors, intensities, and perturbation sizes. Third, we design goal-based and area-based loss functions to achieve high ASRs with minimal perturbations, supporting three attack goals: hiding attack, generative attack, and misclassification attack. To solve the challenge (2), we present several fluorescence-specific transformation methods to improve the robustness of FIPatch in the physical world. The consistency between digital and physical experimental results further validates the effectiveness of our approach.

We still believe our method is novel. In the next version, we will extend the contributions to elaborate on our innovations and highlight the differences from existing methods.

Q-2. The fluorescent patch is visible.

A-2. Please refer to Reviewer 1t4h’s Q-3.

Q-3. The evaluation uses outdated CNN models.

A-3. We follow prior work (e.g., [1] (USENIX-2023), [2] (CVPR-2022), [3] (NDSS-2022), [4] (USENIX-2021), [5] (CVPR-2021), [6] (NDSS 2024)) by using both one-/two-stage detectors and classifiers (including CNN) for TSR.

Q-4. Can the authors include existing physical attacks (e.g., sticker, projector) for a fair comparison?

A-4. It is unfair to compare existing work in the experimental section. Due to the diversity of physical environments, the difficulty of acquiring attack equipment, and the unavailability of source code, most physical attacks [6] (NDSS 2024), [1] (USENIX-2023), [2] (CVPR-2022), [4] (USENIX-2021), [5] (CVPR-2021) are compared with other methods only from a theoretical perspective. In addition, sticker attacks all vehicles indiscriminately once deployed, which lacks flexibility. Although projectors can be actively triggered, the cost of attack is too high, and the price of projectors used in [4] ranges from $1,500 to$44,379. Therefore, we follow the comparison approach adopted in these works.

To address the reviewer’s concern, we establish several baselines for comparison:

• For the hiding attack, we cover the entire surface of the traffic sign (TS) with fluorescent ink.
• For the misrecognition attack, we draw a 14cm-radius fluorescent ink circle at the center of a 60 cm × 60 cm TS.
• For the generative attack, a naive approach is to imitate a real TS. However, this compromises stealth, as humans can easily recognize the fake sign.

The ASR results are presented below:

The results show that the baseline methods yield low ASR, with none exceeding 24%. This proves that such naive approaches are invalid and validates the effectiveness of our method. In the next version, we will include the baseline comparison. If insist, we will also incorporate results from related work for further comparison.

Q-5. The high ASR suggests possible overfitting. How well the method generalizes to different conditions?

A-5. A high ASR does not imply that real-world challenges were overlooked. We conducted experiments in both digital and physical settings, along with extensive ablation studies. Especially in the physical-world experiments, FIPatch consistently achieves high ASR across varying distances, angles, UV lamp powers, and vehicle speeds. In fact, the consistency between our digital and physical experiments validates the effectiveness of our approach. Following your suggestion, we evaluated FIPatch under rainy conditions by simulating rain with a sprinkler. The ASR dropped by only 1%. Additional weather experiments will be included in the next version.

Moreover, prior works have also achieved high ASR. For example, [1] (USENIX-2023) reported an ASR of 96%, and [6] (NDSS 2024) achieved 100%. Therefore, our method does not suffer from overfitting.

Q-6. It should include evaluations with more advanced physical defenses.

A-6. We sincerely appreciate your insightful suggestion! We consider the following physical defenses: camera-LiDAR fusion, optical filters, and multi-camera fusion.

• Camera-LiDAR fusion: Since traffic signs are planar objects, LiDAR cannot capture the content of the sign. As a result, existing TSR systems primarily rely on cameras.

• Optical filters: Our attack remains effective even with optical filters. Since fluorescence emits visible light, only visible-light filters could block it. However, no cameras in current AVs use visible-light filters. Moreover, filtering a single color does not prevent the effectiveness of fluorescent materials in other colors.

• Multi-camera fusion: We evaluated multi-sensor fusion defenses in the physical world. Specifically, we captured the same traffic sign from five different angles (−20°, −10°, 0°, 10°, and 20°) and applied a voting-based decision strategy. The ASR results are shown below:

Experimental results show that multi-sensor fusion can reduce the ASR by at most 5%, demonstrating the strong robustness of FIPatch. Moreover, as illustrated in Figure 5, ASR is less sensitive to changes in angle but is significantly affected by variations in distance. In the next version, we will evaluate additional physical-world defense strategies and provide further discussion on the results.

Q-7. The supplementary images are low resolution (32×32). Was the patch really designed at this size?

A-7. We apologize for the possible misunderstanding about the experimental setup. Our FIPatch is designed for the physical setup. The 32×32 size refers to the image’s resolution on the dataset. In the physical setup, the traffic sign is 60cm×60cm, and the real-world adversarial patch, typically no larger than 14cm, is resized and adjusted using EOT transformations to ensure robustness under various physical conditions. In the next version, we will provide a more detailed explanation of the settings used in the physical world.

Q-8. Moving the method details (L60–70) to the Methods section.

A-8. Thanks for your valuable suggestion! In the next version, we will move the details to the methodology section, especially for the physical implementation.

Reference

[1] Wenjun Zhu, Xiaoyu Ji, Yushi Cheng, Shibo Zhang, and Wenyuan Xu. Tpatch: A triggered physical adversarial patch. In USENIX Security, 2023.

[2] Yiqi Zhong, Xianming Liu, Deming Zhai, Junjun Jiang, and Xiangyang Ji. Shadows can be dangerous: Stealthy and effective physical-world adversarial attack by natural phenomenon. In CVPR, 2022.

[3] Wei Jia, Zhaojun Lu, Haichun Zhang, Zhenglin Liu, Jie Wang, and Gang Qu. Fooling the eyes of autonomous vehicles: Robust physical adversarial examples against traffic sign recognition systems. In NDSS, 2022.

[4] Giulio Lovisotto, Henry Turner, Ivo Sluganovic, Martin Strohmeier, and Ivan Martinovic. SLAP: Improving physical adversarial examples with Short-Lived adversarial perturbations. In USENIX Security, 2021.

[5] Alon Zolfi, Moshe Kravchik, Yuval Elovici, and Asaf Shabtai. The translucent patch: A physical and universal attack on object detectors. In CVPR, 2021.

[6] Takami Sato, Sri Hrushikesh Varma Bhupathiraju, Michael Clifford, Takeshi Sugawara, Qi Alfred Chen, and Sara Rampazzi. Invisible reflections: Leveraging infrared laser reflections to target traffic sign perception. In NDSS, 2024.

[7] Eykholt K, Evtimov I, Fernandes E, et al. Robust physical-world attacks on deep learning visual classification. In CVPR, 2018.

Thanks for your valuable comment. We have explained your questions and modified the corresponding content in the next version according to your suggestions.

Q-1. The method introduced in Section 4.1 for region selection feels unnecessary or redundant.

A-1. We apologize for the possible misunderstanding about Section 4.1. We would like to emphasize that Section 4.1 is indispensable for the following two reasons:

• The final perturbation may extend beyond the region of the traffic sign (TS). For example, a real-world image captured by a camera typically includes both the TS and its background. Direct optimization in such cases may generate perturbations that spread into the background, which cannot be physically realized using fluorescent ink. Additionally, although Figure 15 shows that the central region of the TS exhibits the highest percentage of successful attacks, perturbations with a large radius may still exceed the TS boundary. Therefore, it is essential to apply the mask defined in Section 4.1 to ensure that the perturbation remains strictly within the TS area.
• Section 4.1 significantly benefits the optimization process. In real-world scenarios, it is impractical to manually test a wide range of attack parameters on physical TSs. Thus, we propose a simulation-and-optimization framework to search for optimal attack parameters. The automatic localization in Section 4.1 enables a fully automated attack pipeline, ensuring that the generated perturbation is both deployable and confined to the TS region. As a result, attackers can directly apply the optimized perturbation in the physical world with high attack success rates (ASRs).

We will revise the manuscript to reduce the redundancy and better justify the necessity of Section 4.1 and clearly explain how it improves the attack pipeline.

Q-2. Please clarify the threat model and consider evaluating the attack in a black-box transfer setting to better demonstrate its practical robustness.

A-2. We already evaluated FIPatch’s transferability in Table 5 of the Appendix. We define our attack as a black-box setting in Section 3.2, where the attacker has no access to the internal parameters, architecture, or gradients of the target model. This black-box setting is highly practical in the physical world. As shown in Table 5, the ASRs exceed 85% when transferring to models such as ResNet50, ResNet101, VGG13, and VGG16. The lowest ASR, 69%, occurs when transferring from Inception v3 to MobileNet v2. High ASRs, such as the 95% observed when transferring from ResNet50 to ResNet101, indicate that models with similar architectures are particularly vulnerable. These experimental results demonstrate the robustness of FIPatch across different models. In the next version, we will move these transferability experiments to Section 5 and highlight the black-box setting in the threat model.

Q-3. The authors should consider imposing a hard constraint on the attack area size to ensure the perturbation remains imperceptible to human drivers. As shown in Figure 4, the first two traffic signs are difficult to recognize even for humans.

A-3. In our design (Section 4.3), we have already considered an area-based loss, which introduces the smallest perturbation, to enhance the practicality of FIPatch in the physical world. As shown in Figure 4, our generative attack can fool the TSR system even when the perturbation is only a ring, and a driver will not recognize it as a "STOP" sign. In both Hiding and Misrecognition attacks, the driver can still recognize the original traffic sign. Moreover, for the hiding attack, when the attacker does not trigger the perturbation, the traffic sign still appears as a normal stop sign to human drivers; however, once triggered, the TSR system fails to detect it. It is worth noting that for the Hiding Attack, the perturbation radius tends to be larger. This may be because larger perturbations are required to shift the model's attention away from the original traffic sign. We will clarify and discuss this observation in the next version.

Q1. According to Figure 15, the attack success rate is highest in the central region of the traffic sign (TS), while other regions perform significantly worse. This suggests that the optimization process, if unconstrained, would naturally converge toward placing perturbations near the center of the TS to achieve a successful attack. Given this, I wonder whether the optimization alone — without the region localization step in Section 4.1 — might already be sufficient to discover effective perturbation locations. Have you tried running the optimization without the localization module and evaluated how well the adversarial attacks perform in that case? A comparison would help clarify whether Section 4.1 offers substantial additional benefits over direct optimization.

Q2 I noted that the black-box transferability results are included in Table 5 of the appendix. However, I noticed that Table 5 reports only simulation results, whereas Table 2 presents physical-world performance. My original concern was more specifically about evaluating the black-box setting in the physical world, since that’s ultimately the scenario the attack aims to succeed in. Currently, Table 2 seems to reflect a white-box physical setup.

Q3. Thank you for the explanation. I appreciate that the area-based loss in Section 4.3 is intended to limit the size of the perturbation. However, my concern is specifically about visual imperceptibility to human drivers, not just minimizing the perturbation area. Some of the perturbations shown in Figure 4 still appear quite obtrusive, and in those cases, the attack could arguably be replicated by simpler physical occlusion or vandalism, which weakens the motivation for using adversarial perturbations specifically. Have you considered adding a hard constraint on perturbation size (or a perceptual similarity constraint) to enforce a stronger realism requirement?

We appreciate your positive feedback and are glad that our revisions address your concerns. If there are any remaining issues that need to be resolved, please let us know as we are happy to make further improvements.

We greatly appreciate your constructive suggestions, which we believe could be addressed in the next version.

Q-1. Evaluating the transferability against the unknown threat models is important and necessary in this setting.

A-1. We agree that evaluating transferability under a black-box setting is particularly important. In our experiments, we tested the transferability of FIPatch and reported the results in Table 5 in the Appendix. As shown in Table 5, the attack success rates (ASRs) are above 85% when transferring to models such as ResNet50, ResNet101, VGG13, and VGG16. The lowest ASR is 69%, which occurs when transferring from Inception v3 to MobileNet v2. Notably, models with similar architectures (e.g., ResNet50 to ResNet101) exhibit high transferability, with ASRs reaching up to 95%. These results demonstrate the strong transferability of our attack across various models. In the next version, we will move these results into the main text.

Q-2. For three forms of attacks, all the five threat models should be test.

A-2. We apologize for possible misunderstanding about the experiments. As described in Section 4.3, we define goal-based loss functions according to different attack objectives. For hiding and generative attacks, the loss functions are based on the model’s outputs $Pr(object)$ and $Pr(class)$. Note that $Pr(object)$ is the confidence that an object exists in a given cell. Since image classifiers do not produce this objectness score, it is not feasible to evaluate hiding or generative attacks on classification models. Following your suggestion, we have revised Table 2 to include the ASRs of misrecognition attacks against object detectors. The updated results are shown below:

Table: The ASR of FIPatch on various models in the physical world under misrecognition attack.

We will clarify this distinction in the revised manuscript and incorporate the updated results into Table 2. Furthermore, we evaluated the performance of FIPatch across different models in our ablation study, which further highlights its effectiveness and robustness.

Q-3. This paper only verified the proposed method in traffic sign recognition task, how about other object detection tasks, like face recognition, general object detection, etc.

A-3. We chose traffic sign recognition because it is a highly representative task that involves both classification and detection. It is reasonable to extend our method to other scenarios, such as human recognition in surveillance systems, where an attacker could wear clothing with fluorescent materials. As for face recognition, it presents new challenges due to the complex 3D surface of the human face, and we consider it a promising direction for future work. We will include a discussion on the potential extension to other tasks in the next version.

Q-4. The contributions should be extended, thus readers can better understand the novelty in this paper. At least the second contribution should point out what is the specific novelty for the proposed method compared with the current methods.

A-4. We are the first to propose the use of fluorescent ink for physical adversarial attacks. Existing methods can be categorized into those using stickers, light, or acoustic signal. However, sticker-based approaches [1][2][3] suffer from poor stealthiness and indiscriminately attack all vehicles once deployed. Visible light-based attacks [4][5] make it easy to trace the attacker, while invisible light (e.g., infrared [6]) can be easily countered by infrared filters. In addition, acoustic signal-based attacks [7] can be easily blocked by physical signal protection mechanisms. Unlike existing methods, our approach introduces fluorescent ink as a novel attack medium, which significantly enhances stealth and poses greater challenges for defense mechanisms.

Although fluorescent ink enables invisible and actively triggered features, designing an effective and robust physical adversarial patch remains non-trivial with the following challenges. First, it is challenging to simulate fluorescent effects and determine the optimal choices of massive factors in fluorescent ink, e.g., color, transparency, and size, for achieving high attack effectiveness. Second, fluorescent effects are easily influenced by real-world environments, including surroundings, ambient light, and vehicle distance. Existing methods do not address these challenges, and most require manual annotation of traffic sign locations.

To address the above challenges, we design the FIPatch that leverages the unique properties of fluorescent ink. In more detail, our methodology consists of four key modules. First, we develop a color-edge fusion method to automatically locate traffic signs, enabling precise application of fluorescent ink to the signs, rather than invalid backgrounds. Second, to effectively simulate fluorescent effects, we model fluorescent perturbations on traffic signs by defining the various critical parameters of fluorescent ink, including colors, intensities, and perturbation sizes. Third, we design goal-based and area-based loss functions to achieve high ASRs with minimal perturbations, supporting three attack goals: hiding attack, generative attack, and misclassification attack. Finally, to improve the robustness of FIPatch in the physical world, we present several fluorescence-specific transformation methods that simulate fluorescence perturbations for real-world attacks.

The consistency between digital and physical experimental results further validates the effectiveness of our approach. In the next version, we will extend the contributions to elaborate on our innovations and highlight the differences from existing methods.

Reference

[1] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In CVPR, 2018.

[2] Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. In WOOT, 2018.

[3] Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual-sensitive gan for generating adversarial patches. In AAAI, 2019.

[4] Giulio Lovisotto, Henry Turner, Ivo Sluganovic, Martin Strohmeier, and Ivan Martinovic. SLAP: Improving physical adversarial examples with Short-Lived adversarial perturbations. In USENIX Security, 2021.

[5] Ranjie Duan, Xiaofeng Mao, A Kai Qin, Yuefeng Chen, Shaokai Ye, Yuan He, and Yun Yang. Adversarial laser beam: Effective physical-world attack to dnns in a blink. In CVPR, 2021.

[6] Takami Sato, Sri Hrushikesh Varma Bhupathiraju, Michael Clifford, Takeshi Sugawara, Qi Alfred Chen, and Sara Rampazzi. Invisible reflections: Leveraging infrared laser reflections to target traffic sign perception. In NDSS, 2024.

[7] Wenjun Zhu, Xiaoyu Ji, Yushi Cheng, Shibo Zhang, and Wenyuan Xu. Tpatch: A triggered physical adversarial patch. In USENIX Security, 2023.