FFAM: Feature Factorization Activation Map for Explanation of 3D Detectors

We greatly appreciate your insightful feedback. Below please find our clarifications in response to your comments.

Q1. The only concern is about the application, or the value in applications, of this proposed method. I understand this is a good adaptation and attempt to apply such explanation methods on LiDAR-based 3D detectors, but it is still unclear how it can benefit the procedure of improving 3D detectors or downstream applications. I would be more interested in how it produces a stronger 3D detector or a 3D detector with better controllability and safety, especially for safety-critical scenarios like autonomous driving.

Authors’ Reply: We fully agree with your viewpoint. Our FFAM can be used to find the regions of interest of detectors on point cloud input. It also can provide visual explanation for different object attributes at a fine-grained level. In section 4.3, we utilize FFAM to reveal the detection mode of false positive predictions generated by detectors. There are three main observations in the experiment. First, we observe the average saliency maps of false positives exhibit similarities to those of true positives. The detector predicts a false positive because it detects a similar pattern to that of a true positive. Second, false positives tend to be surrounded by more noise points, with a point density of approximately one-third of true positives. We believe noises and sparse density may be significant factors contributing to the occurrence of false positives. Lastly, the ratio of car, pedestrian, and cyclist objects in true positives is approximately 36:5:2, while in false positives, it is 13:8:2. This suggests car objects are less prone to false positives compared to pedestrian and cyclist objects. We think that the above observations can help researchers to design more efficient and reliable 3D object detectors. We believe there are more applications of FFAM that can be explored. In our future work, we will use FFAM to improve the performance (including accuracy and speed) of 3D detectors.

Q2. In addition, there are also related areas or applications, such as 3D detection in indoor scenes (ScanNet, SUN RGB-D), etc, and camera-based/multi-modality 3D detection. I am also curious about the performance of such explanation methods when applied to those problems. How about the performance, and is there any new challenge?

Authors’ Reply: In this paper, our main focus is on explaining LiDAR-based 3D detectors on outdoor scenes. For indoor scenes or other modalities as input, as long as the intermediate features of the network can be directly linked to the input, our FFAM can be applied effectively. We believe that the most challenging task would be explaining 3D detectors based on multi-view cameras. This is because the correspondence between the intermediate features of such detectors and the input is relatively intricate. Many of them employ learning-based methods to elevate two-dimensional images into three-dimensional space.

We greatly appreciate your insightful feedback. Below please find our clarifications in response to your comments.

Q1. Real-Time Applicability: How does the computational overhead of FFAM compare to existing methods in real-time applications, especially in autonomous driving scenarios?

Authors’ Reply: Compared to the existing method OccAM, our approach has a significant advantage in speed. NMF and voxel upsampling are implemented by CUDA operators which could use GPU to accelerate the process. The main overhead of our FFAM lies in gradient backpropagation. Therefore, the delay of FFAM depends on the size of the model being explained. Currently, the backpropagation delay of most 3D object detectors is around 100ms. On the other hand, the existing method OccAM is a perturbation-based explanation method, which requires extensive sampling of inputs and then processing them one by one. As a result, this process is very slow. Typically, using OccAM to obtain an visual explanation requires several minutes. However, visual explanation methods (including our FFAM and OccAM) belong to post hoc explanations of the model. They are not used in real-time applications of autonomous driving. Therefore, we did not compare the speed of these methods in the paper.

Q2. How sensitive is FFAM to the choice of detector? We noticed that two kinds of detectors were used in this paper. More comparison and analysis are necessary.

Authors’ Reply: FFAM is not specific to detector architecture. It can be widely applied to existing LiDAR-based 3D detectors. In Appendix A.2, we apply FFAM to some other state-of-the-art detectors, such as DCDet, PV-RCNN, and Voxel R-CNN.

Q3. Can FFAM be extended to other 3D detectors or modalities, such as RGB-D sensors or radar data? If so, what modifications would be necessary?

Authors’ Reply: FFAM can be extended to extensive 3D detectors. In this paper, our main focus is on studying point clouds as the input format. FFAM has not yet been applied to other modalities. But we believe that RGB-D and radar data can be converted into point cloud format, so these modalities should also be feasible for FFAM to process.

Q4. FFAM requires access to the feature maps within 3D detectors, which may not always be possible, especially for proprietary or closed-source systems.

Authors’ Reply: We concur with your observation regarding the limitation in FFAM. We also pointed out this limitation of FFAM in the conclusion section. We believe that FFAM can primarily be used to provide researchers with some insights to improve detectors. In addition, it can also be used to reveal the internal working mechanism of detectors to users, thereby improving their understanding and trust in detectors.

Dear reviewer v6wE, is there anything more you would like to ask of the authors, before the author-reviewer discussion period ends (tomorrow)?

We greatly appreciate your insightful feedback. Below please find our clarifications in response to your comments.

Q1. The rationale for using NMF is unclear. It appears to be a learning-based, PCA-like method to extract saliency from voxel features. How does it compare to the proposed method on the global activation map?

Authors’ Reply: Non negative matrix factorization (NMF) is a matrix factorization technique used to discover potential concepts in features. The process of NMF is to approximately decompose a non negative matrix $V$ into the product of two non negative matrices H and W, i.e. $V ≈ HW$. Among them, $W$ and $H$ are non negative, which makes NMF more interpretable in many fields. For example, in facial recognition, the basis vectors in the $W$ matrix usually represent specific concepts such as nose, eyes, mouth, etc [1]. In our work, we utilize NMF to uncover latent concept within voxel features of 3D detectors. Typically, voxel features with effective detection clues in 3D detectors contain richer semantic concepts. Therefore, we could sum the activation coefficients of different concepts in $H$ matrix to obtain a global activation map. The basis vectors obtained by PCA do not have clear semantic concepts. Therefore, we have not compare our method with PCA-like method.

Q2. There is a lack of ablation studies, such as those examining the parameter $r$.

Authors’ Reply: Due to page limitations in the main body of the paper, we have included the hyperparameter analysis and ablation experiments in the appendix. Please refer to Appendix A.1 and A.4 for details.

Q3. The gradient weighting method is intriguing and seems sensible. Is this method original to your work?

Authors’ Reply: As far as we know, we are the first one that use the gradient to refine a global activation map. The previous method that is closest to our method is ODAM. It also utilizes backward gradients to generate saliency maps. However, there are two main difference between our method and ODAM. Firstly, the usage of gradients is different. ODAM multiply the gradient map with mid feature maps, and then sum the values along the channel dimension to obtain the final saliency maps. While our FFAM utilizes the gradients to generate a weighting item as follows:

where $G_{\cdot k}$ refers to the $k$-th channel of gradient map $G$, and $d$ denotes the number of channels. Secondly, ODAM is designed to generate visual explanations for image detectors. While our FFAM is used to explain LiDAR-based 3D detectors.

Q4. It would be beneficial to use $W$ as a weight in Equation 2. The current version is a little confusing.

Authors’ Reply: Equation 2 represents the process of solving for NMF. Since it is difficult to find a numerical solution for the non-negative matrix factorization of matrix $A$, only an approximate solution can be obtained. We attempt to find a matrix $\hat{A}$ that closely approximates $A$, while also satisfying the product of two non-negative matrices $H$ and $W$.

Q5. In Section 4.1, it is claimed that the saliency map generated by FFAM is superior to OccAM because it is more focused on the object. Please provide further validation on why a clearer activation map is considered better.

Authors’ Reply: In the context of visual explanation for 3D detectors, a clearer activation map is considered superior because it aids in better identifying the region of interest within the point cloud. We have conducted quantitative experiments to validate our method and previous explanation methods. As shown in Table 1 and Table 3, our FFAM achieves best results on VEA, PG and enPG metrics. These metrics reflect the degree of focus of an explanation method on an object. Furthermore, our FFAM performs best on the Deletion and Insertion metrics which are are widely used to evaluate explanation methods. As shown in Figure 5, our methods have the fastest performance drop and largest increase for Deletion and Insertion respectively, showing points highlighted in our saliency maps have a greater effect on detector predictions than the other methods.

Q6. It would strengthen the paper if an example could be provided demonstrating how FFAM can help identify some non-trivial error modes and offer insights for improving the detector.

Authors’ Reply: In section 4.3, we utilize FFAM to find the modes of false positives generated by a detector. There are three observations in the experiments. We believe these observations will provide some insights for researchers to improve 3D detectors. First, we observe the average saliency maps of false positives exhibit similar similarities to those of true positives. The detector predicts a false positive because it detects a similar pattern to that of a true positive. Second, false positives tend to be surrounded by more noise points, with a point density of approximately one-third of true positives. We believe noises and sparse density may be significant factors contributing to the occurrence of false positives. Lastly, the ratio of car, pedestrian, and cyclist objects in true positives is approximately 36:5:2, while in false positives, it is 13:8:2. This suggests car objects are less prone to false positives compared to pedestrian and cyclist objects.

[1] Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. nature, 1999.

Dear reviewer bytz, is there anything more you would like to ask of the authors, before the author-reviewer discussion period ends (tomorrow)?

We greatly appreciate your insightful feedback. Below please find our clarifications in response to your comments.

Q1. The description of Non-negative Matrix Factorization (NMF) is unclear. When the point cloud is large in scale, NMF may become unstable and exhibit a long convergence time.

Authors’ Reply: In our experiments, Non-negative Matrix Factorization (NMF) maintains excellent stability and rapid convergence times across different scales of point cloud, encompassing those from the KITTI and Waymo Open datasets. The implementation of NMF is facilitated by well-established libraries, which can be seamlessly integrated with PyTorch code.

Q2. The core innovation of this paper lies in feature factorization. The authors directly chose NMF but did not provide a reason for this choice. For instance, Principal Component Analysis (PCA) can also achieve feature factorization.

Authors’ Reply: We agree with your perspective that Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF) are both effective tools for feature decomposition and dimensionality reduction. However, the basis vectors derived from PCA can not represent a clear semantic concept, and thus are primarily used for data dimensionality reduction. In contrast, the basis vectors of NMF exhibit non-negativity and additivity, often representing specific concepts such as car doors and wheels. The DFF [1] is the first work that employs NMF to localize semantic concepts within images. Inspired by DFF, we utilize NMF to reveal the latent semantic concepts embedded in the intermediate voxel features of 3D object detectors. Typically, voxel features that provide effective cues for detection in 3D detectors possess richer semantic concepts, which can be used to create saliency maps.

Q3. The object-specific gradient is element-wise multiplied with the global concept activation map to obtain the specific activation map of the object. In other words, the object-specific gradient inhibits the activation that do not belong to the current object. Therefore, it seems reasonable to directly use the weighting of the object-specific gradient as the activation maps.

Authors’ Reply: We agree with your point that object-specific gradient can serve as the activation map. However, it only highlights the object-specific region in a point cloud and is struggle to differentiate the importance of point within the object-specific region. NMF can be used to help uncover the recognition pattern within the object-specific region. As shown in Figure 4, NMF assists in identifying the detector's distinct recognition patterns for various categories and object attributes. The quantitative results in Table 9 emphasize the enhanced performance achieved by integrating the object-specific gradient with NMF. This confirms the significant role of NMF in obtaining more nuanced and detailed visual explanations.

Q4. Since the gradient map G can already represent the current object, why not perform NMF on the gradient map and then use a method similar to obtaining a global activation map to derive the object activation map?

Authors’ Reply: The raw point features contain substantial semantics which is beneficial for NMF to uncover latent concepts within these features. However, the gradient map G only contains backward gradients for a specific object. Consequently, employing NMF to extract concepts from gradient maps lacks a clear conceptual basis.

Q5. Object-Specific Gradient Weighting is a general module. Can it be applied to other methods of generating activation maps, such as OccAM?

Authors’ Reply: we believe that incorporating object-specific gradient weighting into other methods is feasible. But we haven't tried combining the weighting module with other methods yet. Because other methods usually have their own approaches of obtaining object-level visual explanations. For example, OccAM is a perturbation-based method that involves randomly masking the input point cloud to assess performance changes in the output. As it inherently serves as an object-level explanation method, there is no immediate necessity to integrate it with object-specific gradient weighting. Consequently, we have not attempted to combine these techniques.

Q6. Regarding voxel upsampling, the authors provide a limited description. I am curious about why they chose the Gaussian kernel and how it compares to other upsampling methods such as trilinear interpolation and transpose convolution.

Authors’ Reply: Given that voxels are sparsely scattered throughout 3D space, identifying all neighboring points for a given point to be interpolated presents a challenge. Consequently, trilinear interpolation is not well-suited for handling sparse voxels. On the other hand, transpose convolution, being a learning-based technique, does not align as the optimal choice within our framework. In contrast, our voxel upsampling method initiates by searching for neighbors within a specified range and subsequently applies a Gaussian kernel to weigh them. This method is more adaptable for our FFAM.

[1] Edo Collins, Radhakrishna Achanta, and Sabine Susstrunk. Deep feature factorization for concept discovery. In ECCV, 2018.

Dear reviewer 93Lo, is there anything more you would like to ask of the authors, before the author-reviewer discussion period ends (tomorrow)?

Dear reviewer 93Lo, do you have any further questions regarding my response?