Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRF
We introduce a novel approach that modeling adversarial examples as NeRFs. Our examples exhibit better attack performance and show promising transferability in the context of 3D object detection.
摘要
评审与讨论
The paper introduces Adv3D, a method for generating 3D adversarial examples for 3D object detection in driving scenarios. The authors propose to model adversarial examples as Neural Radiance Fields (NeRFs) and train the adversarial NeRFs by minimizing the confidence predicted by 3D detectors. They also propose primitive-aware sampling and semantic-guided regularization to enhance the physical realism and effectiveness of the adversarial examples. The paper evaluates Adv3D on the nuScenes dataset and demonstrates its effectiveness in causing a significant performance reduction in 3D detectors.
优点
(1) The paper addresses an important and practical problem of adversarial attacks on 3D object detection in driving scenarios, which has significant safety implications for autonomous driving systems.
(2) The use of NeRFs for modeling adversarial examples is innovative and provides more realistic and realizable attacks compared to traditional 2D pixel attacks.
(3) The proposed primitive-aware sampling and semantic-guided regularization techniques enhance the physical effectiveness and realism of the adversarial examples.
(4) The experiments on the nuScenes dataset demonstrate the effectiveness and transferability of Adv3D in different poses, scenes, and 3D detectors.
缺点
(1) The paper lacks detailed implementation details, making it difficult to reproduce the study. The authors should provide more information on the specific architectures, hyperparameters, and optimization methods used in training the adversarial NeRFs.
(2) The evaluation of MESH ATTACK should discuss some existing papers[1,2].
(3) The paper could benefit from a clearer and more structured presentation of the proposed methodology and experimental results. Some parts of the paper are difficult to follow, and additional clarity would improve the reader's understanding.
(4) The detailed implementation details of adv training should be provided.
[1] Isometric 3d adversarial examples in the physical world, NIPS 2022
[2] 3d adversarial attacks beyond point cloud, Information Sciences 2023
问题
/NA
伦理问题详情
/NA
The study focuses on creating adversarial attacks on 3D object detectors using instance-level NeRFs. They begin with a vehicle representation using a NeRF, which determines both its shape and texture. This vehicle is then rendered into an image and merged into the original image through a copy-paste method. This modified image is then used to challenge 3D object detectors. The feedback from these detectors is then employed to further refine the NeRF, but only its texture. The tests confirm that their adversarial samples effectively challenge a range of 3D object detectors. Moreover, when these detectors are trained using these adversarial examples, they not only become more resistant but their overall performance also enhances.
优点
The work is clearly presented and easy to understand. The analysis offers valuable insights, especially in Section 5.3 where the robustness of the 3D detector architecture is discussed, and in Section 5.4 where it's shown that adversarial training can improve performance. Additionally, the experiments cover a range of architectures, which adds depth to the study.
缺点
My major concern is that whether the formulation of NeRF is necessarily, from the motivation perspective. The optimization is essentially finding the color, density of the volume. However, I believe most vehicle objectives are not translucent; the optimized 3D object is very hard to realize. This is evident as authors need to improve the physical realizability.
So we are missing a baseline here: optimizing the surface texture as a 3D mesh, using existing differentiable mesh renderers (such as Neural Mesh Renderer). The latter is easier to optimize (2D texture space), and more physically realizable (because it is a texture map rather than a volume). The authors said it "enables patch attacks in a 3D-aware manner by lifting the 2D patch to a 3D box", so we really need a baseline to showcase such lifting is necessary.
The practicality of this attack is also questionable because in the supplementary material, the mini-nature attack does not seem to be very successful.
In general, my decision largely depends on the first point: the NeRF representation may not be necessary under the current settings. Optimizing the texture image should just work; such volume formulation will make it harder to physically realize and does not bring much benefit other than differentiable rendering.
问题
Please see the weakness section.
Dear reviewer 7cQe,
We would like to thank you for the detailed feedback and for appreciating the novelty of our idea and clear writing. In the following, we reply to individual questions and comments raised by the reviewer:
Physical realizability of NeRF. Our method defines the NeRF volume in SDF (Signed Distance Field) space, converging the volume to a surface area on the zero-level set, thus the volume has a meaningful definition. Our real-world experiments (please see supp) also demonstrate that the adversarial texture can be created in the real world and display satisfactory attack results.
Mesh baseline. We have provided a comparison with a mesh baseline that is shown in Sec.5.1. We urge the reviewer to check it out.
Thank you!
This paper studies how to generate 3D adversarial examples for 3D object detection in autonomous driving. The authors propose to use Neural Radiance Fields (NeRFs) to model adversarial examples, and they train the adversarial NeRF by minimizing the surrounding objects’ confidence predicted by 3D detectors on the training set. The authors also propose primitive-aware sampling and semantic-guided regularization that enable attacks with camouflage adversarial texture.
优点
-
This paper studies how to generate adversarial examples for 3D object detection in autonomous driving, which is an important problem for enhancing the safety of autonomous vehicles.
-
The authors consider six detection models when evaluating the performance of the proposed attack.
缺点
-
The advantage of the proposed method over existing attacks is not clear. The authors claim that implementing existing attacks is challenging because their adversary must have direct contact with the attacked object. However, it is not clear why having direct contact with the attacked object is challenging. I do not think placing an object on top of a vehicle is a challenging task. In addition, many existing methods can be used to attack multi-sensor fusion, which has been widely adopted by today’s autonomous vehicles. What’s the advantage of the proposed attack compared to those existing attacks?
-
The authors do not describe the threat model. The information about the detection model and the victim vehicle that the attacker can access remain unclear. How can the attacker obtain such information in practice?
-
The practicability of the proposed attack is questionable. To achieve the attack goal, the car with the patch must assume a particular pose and appear in an unusual location on the road (as shown in Figure 3(b)). The car itself may cause traffic accident in practice.
-
The authors do not evaluate the proposed attack in real-world driving environments. The influence of view angle, lighting conditions, and vehicle speed on the effectiveness of the attack remains unclear.
问题
-
What’s the advantage of the proposed attack compared to existing attacks?
-
What’s the threat model? How can the attacker obtain the information about the detection model and the victim vehicle in practice?
-
Does parking a car on the road as shown in Figure 3(b) and Figure 3(c) appear suspicious in the real world?
-
Could the view angle, lighting conditions, and vehicle speed influence the effectiveness of the proposed attack?
This project proposes modeling adversarial examples for 3D detection in driving scenarios using Neural Radiance Fields. Specifically, Adv3D takes an existing NeRF model which composites generated views into 2D scenes (Lift3D), and then optimize the texture latent codes of this NeRF model to find an adversary which will result in surrounding 3D objects not being detected by 3D detectors. The goal is to find adversaries which can transfer across object poses, and without altering the shape which makes it possible to print the textures making them physically realizable. The paper also explores the impact of enforcing that the composited texture be overlayed onto semantic parts of cars, including doors, side, etc. Finally, the paper investigates the utility of pre-training with such data and evaluating robustness to such 3D attacks.
优点
-
Transferability is key: Very often adversarial examples are brittle, i.e., they attack a particular model, and a particular dataset. This is problematic for two reasons---(1) Adding them to pre-training doesn't help much as it only improves the model it originally attacked, (2) Such narrow attacks tell us very little about the underlying principles driving such attacks. Thus, broad-spectrum, transferrable attacks are immensely important.
-
Preliminaries well written: Would make it very easy for even a novice reader to understand, and follow the work.
-
NeRF + Adversarial is interesting: NeRF's offer a unique opportunity thanks to their photorealism + differentiability. Work on this intersection is interesting.
-
Focussing on just texture latent codes makes it possible to print out and test the impact of such attacks in the real world. Such physical realizability is very useful in studying the real-world impact.
缺点
- Missing literature: Several important related threads of work are not present, which doesn't place this work in the right context of existing literature. These include:
-
NeRF + Adv: There already exist some works which have explored this intersection. It would be helpful to include these, and to talk a bit about how this work differs from these. https://proceedings.neurips.cc/paper_files/paper/2022/hash/eee7ae5cf0c4356c2aeca400771791aa-Abstract-Conference.html, https://openaccess.thecvf.com/content/ICCV2023W/AROW/html/Horvath_Targeted_Adversarial_Attacks_on_Generalizable_Neural_Radiance_Fields_ICCVW_2023_paper.html among others.
-
Differentiable Rendering: Beyond NeRF and neural network based image generation, there have been recent advances in differentiable physically-based rendering which is accurate, and highly photorealistic. Would be good to add these citations. https://dl.acm.org/doi/pdf/10.1145/3414685.3417833, https://people.csail.mit.edu/tzumao/diffrt/, https://research.nvidia.com/labs/rtr/publication/bangaru2023slangd/bangaru2023slangd.pdf, https://inria.hal.science/hal-02497191/document.
-
Other papers on 3D adversarial attacks and viewpoint generalization: https://arxiv.org/abs/2106.16198, https://arxiv.org/abs/1808.02651, https://www.nature.com/articles/s42256-021-00437-5.
- Writing/Presentation of Methods unclear: There are several details in the Methods section that are very hard to follow and ambiguous. Correcting these would be imperative to make sure the paper is understandable:
- "Pose of an adversarial example": An adversarial example in this case refers to an image with the composited NeRF object (+texture), and the original 3D objects from the nuScenes dataset. Which object's pose are you referring to? It might help referring to these objects separately---the existing ones, and the NeRF object added using Lift3D. And then specify what object the sampled pose refers to.
- Figure 1 shows a loop which is optimized over iterations. It seems that the thing being optimized is the adversarial texture patch composited onto the image. However, both the patches shown in Figure 1 (top and bottom row) are the same which makes it unclear what got optimized in the iteration? Following from above: was the pose sampled fro the red car, or the grey car?
- "EOT by average loss over whole batch": It is not specified what is contained in a batch. The idea behind the original EOT concept was to ensure transferability of attack across different transformations. In this case, are we averaging over different poses of the added red car, or the original grey car? How is this ensured that the batch actually samples the correct distribution w.r.t. which the expectation needs to be calculated?
- What are primitives of adversarial examples? It isn't clearly defined what the paper refers to as primitives.
- Experiments very weak:
-
No statistics reported: The closest comparison to NeRF attacks are Mesh ones. Firstly, only one baseline is provided, and there are no error bars or tests for statistical significance reported. This is also true for all experiments.
-
No real baselines: The experiments are very thin, with hardly many numbers reported.
(a) Is Clean referring to no attack? What if no adversarial patch is added, but the output of Lift3D is optimized? If the attack is good enough, it would reduce the utility of the adversarial patch.
(b) The patch is added by compositing the image. Was it investigated how much of this adversarial attack was due to NeRF and how much due to just the Image compositing? An easy baseline would be taking the size of the patch, and optimizing pixels in the original 2D image directly without Lift3D to see if a similar sized patch can be found which can break the detection networks.
(c) How do results vary with optimizing the shape latent codes of the NeRF as opposed to texture? (2) (3) How does semantic part normalized by surface area impact?
(d) The results in Fig 4 on viewpoint perfectly correlates with surface area of object if sampling was uniform---is this an outcome of the viewpoint, or just of the number of pixels replaced by compositing?
(e) How do results in Fig4(b) compare with the distribution of viewpoints shown during training? Is there correlation? If so, it is only an artifact of what was shown during training. Same is true for location.
(f) How does this compare to other general purpose NeRF + adversarial attack approaches?
(g) Why are results reported with only nuScenes? There exist several outdoor driving datasets.
问题
Please refer to the weaknesses to see the questions.
Overall: This work focusses on an interesting problem, with very important real-world implications. However, in its current form the manuscript is not ready for publication. The missing literature and writing would be easy to fix, and would make the manuscript much stronger. However, the lack of detailed experiments and rigorous baselines means this work is currently not mature enough for publication. If these can be addressed and the work made more rigorous, I believe this work can be of value to the community.