Directional Distance Field for Modeling the Difference between 3D Point Clouds

We appreciate your acknowledgment of our work and the constructive feedback you've provided. In the subsequent discussion, we will thoroughly address the concerns you have raised.

Comments 1. This work should add comparisons against relevant shape or surface descriptors in the literature.

Response: The reviewer may have misunderstood our Directional Distance Field (DDF) to some extent, resulting in the comparison with the mentioned surface descriptors not making sense. The detailed reasons are listed as follows.

• The reviewer mentions the 3D Shape Context as a surface descriptor, which employs histograms to characterize local point cloud structures. However, this descriptor is non-differentiable, making it unsuitable for replacing our differentiable distance function in the distance metric. Additionally, other differentiable descriptors, including learning-based ones, are not viable alternatives for the distance metric. This limitation essentially arises from these descriptors explicitly characterizing local surface structures, defined on surfaces where point clouds are sampled. In contrast, our DDF serves as an implicit field, representing underlying surfaces as its iso-surface with a specific value. Consequently, the DDF is defined across the entire 3D space, allowing it to be used at any point.
• The key points used in the surface descriptor differ from the reference points used in our method. Key points, usually a subset of the whole point cloud, are selected to reduce computational complexity while retaining crucial information. And the correspondence between key points in different clouds is established through feature matching. Differently, the reference points in our method are located near the underlying surfaces and they are used to compute the difference between the DDFs of the point clouds, where the correspondence is established naturally according to these shared reference points.

Comment 2. Another weakness is in formulations that should be presented more clearly, perhaps improving notation. For example, it is unclear why "g(qm, P1) and g(qm, P2) share identical weights". From equations, they seem to be different as calculated over different point clouds.

Response: As stated in Sec. 3.3, the points within $\Omega(\mathbf{q},\mathbf{P}_1)$ and $\Omega(\mathbf{q},\mathbf{P}_2)$ are first sorted based on their distances to the reference point $\mathbf{q}$, i.e., $||\mathbf{p}{\tiny{1,1}}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny{1,K}}-\mathbf{q}||$ with $\mathbf{p}{\tiny{1,1}},...,\mathbf{p}{\tiny{1,K}}\in\Omega(\mathbf{q},\mathbf{P}_1)$ and $||\mathbf{p}{\tiny{2,1}}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny{2,K}}-\mathbf{q}||$ with $\mathbf{p}{\tiny{2,1}},...,\mathbf{p}{\tiny{2,K}}\in\Omega(\mathbf{q},\mathbf{P}_2)$. This sorting process facilitates the transfer of weights from the target/ground-truth point cloud to the source/generated cloud. Specifically, for the calculation of directional distances across both point clouds, we adhere to the relation $w(\mathbf{q},\mathbf{p}{\tiny{2,k}})=w(\mathbf{q},\mathbf{p}{\tiny{1,k}}),\ k=1,...,K$. The weights can be computed from $\mathbf{P}_1$ (resp. $\mathbf{P}_2$) and then shared with $\mathbf{P}_2$ (resp. $\mathbf{P}_1$), according to the task.
Our weight-sharing strategy is built upon the observation that if two point clouds are exactly the same, the weights obtained through the same calculation method should be equal for a typical reference point. If the weights are derived independently using the inverse distance for two point clouds, for the point cloud to be optimized/generated, the directional distance will be a high-order non-linear function of its points, thereby complicating the optimization process. The experimental results in Table 7 of our manuscript show the superiority of the shared weights.
Eqs. (1) and (2) represent the calculation of our designed DDF. In Section 3.4 of the updated pdf file, we clarified the motivation of the shared weights.

Comment 3. In Sec. 4.2, R should be in SO(3). Also, there is some abuse of notation when applying a rotation over a point cloud.

Response: In Section 4.2, we stated that '$\mathbf{R}\in\mathbb{R}^{3\times 3}$ is the rotation matrix', and this is equivalent to '$\mathbf{R}\in`SO`(3)$'. As the reviewer rightly noted, the latter representation is more concise. We updated the manuscript with this form.
Thanks to the reviewer for the reminder. We have identified an inaccurate expression in the matrix multiplication $\mathbf{R}\mathbf{P}{\tiny src}$ in Eq. (5) of the manuscript, where $\mathbf{R}\in\mathbb{R}^{3\times 3}$ and $\mathbf{P}{\tiny src}\in\mathbb{R}^{N_{\small src}\times 3}$. To rectify this, we introduce a new symbol to denote the rigid transformation on the point cloud in Eq. (5) of the revised manuscript, denoted as $\mathcal{T}(\mathbf{P}{\tiny src},\mathbf{R},\mathbf{t})$.

We appreciate your acknowledgment of our efforts and constructive comments. In what follows, we address your comments comprehensively.

Comment 1. The name directional distance field is not suitable. I do not understand what is it until I finished reading the method section. The 'field' often indicates the signed distances or occupancies learned by a neural network. The proposed loss to measure distances from a reference point to the underline surface is not a 'field'.

Response: The field can also represent implicit fields calculated by classical methods, not limited to neural networks, such as IMLS [1], where Signed Distance Fields are estimated from point clouds using classical techniques without neural networks. Hence, utilizing 'Field' in our implicit representation of underlying surfaces is appropriate. \ The main idea of our distance metric is measuring the disparity between point clouds through the difference between the DDFs estimated from them, thus we denote our proposed DDF-based distance metric as 'DDF'.
[1] Provably good moving least squares.

Comment 2. The presentation can be improved. I suggest that the authors to improve the writings in the introduction to make the readers understand the loss more easily. The inappropriate name 'directional distance field' and the unexplained 'reference point' make the introduction not clear enough. I can not understand the loss until I finish reading the method section, but finally I find the loss quite simple.

Response: At the start of Section 3, we include a figure in the revised manuscript to visually depict the overall concept of our distance metric, enhancing clarity and comprehension.

Comment 3. In Fig. 3, I find that DCD achieves quite good performances, why is the quantitative results of DCD in Tab. 1 that bad? More comparisons are needed.

Response: Table 1 of the manuscript shows the averaging numerical results of each category containing many 3D point clouds rather than only those displayed in the figure. In the updated manuscript, additional visual results have been included in Figure 11 within Appendix B. Furthermore, histograms depicting various evaluation metrics for different methods are presented in Figure 12, offering insights into the distribution of error and accuracy associated with each method.

Comment 4. As shown in Fig.4, DDF is less efficient than CD and DCD. What is reason? Since CD measures point-to-point distances which should be much slower.

Response: In our metric, we introduce a set of reference points whose quantity exceeds the number of points in the point clouds. This allows us to indirectly establish correspondences between two point clouds.Calculating the DDFs at the position of each reference point involves K-NN searching, which constitutes the most time-consuming aspect of our method. Tables 9 and 10 in our manuscript illustrate the relationship between the running time and the number of reference points, as well as the KNN size. Unlike CD and DCD, which establish correspondence for each point in the point cloud by finding the 1-NN point in the other point cloud, our method operates slightly slower. However, when considering both efficiency and effectiveness, our metric outperforms CD, DCD, and EMD.

Comment 5. It will be interest to see the performance of DDC under more downstream tasks like point cloud completion, point cloud generation, etc.

Response: Thank you for your valuable suggestion. Due to time constraints, we have conducted a preliminary exploration of the application of our distance metric in the point cloud completion task. We utilized the widely used framework PCN [2], in which we substituted the CD with our distance metric. We maintained the same experimental settings as the original PCN and selected several categories from the ShapeNet dataset for our experiments, namely Airplane, Car, Table, and Chair. The following table lists the numerical results, where it can be seen that the results from the network trained through our distance metric achieve higher F-Score than those from the network trained by CD.

[2] PCN: Point Completion Network.

Thank you for acknowledging our work and offering constructive comments. We will respond comprehensively to your comments as follows.

Comment 1. The reviewer is confused by the shared identical weight operation. The reference points are generated from one of the two point clouds, and the weights in Equ.1 are computed for each kNN points and reference points. The reviewer wants to ask how to share the weights in g(q_m, P1) and g(q_m, P2)?

Response: As stated in Section 3.3 of the manuscript, the points within $\Omega(\mathbf{q},\mathbf{P}_1)$ and $\Omega(\mathbf{q},\mathbf{P}_2)$ are first sorted based on their distances to the reference point $\mathbf{q}$, i.e., $||\mathbf{p}{\tiny 1,1}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny 1,K}-\mathbf{q}||$ with $\mathbf{p}{\tiny 1,1},...,\mathbf{p}{\tiny 1,K}\in\Omega(\mathbf{q},\mathbf{P}_1)$ and $||\mathbf{p}{\tiny 2,1}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny 2,K}-\mathbf{q}||$ with $\mathbf{p}{\tiny 2,1},...,\mathbf{p}{\tiny 2,K}\in\Omega(\mathbf{q},\mathbf{P}_2)$. This sorting process facilitates the transfer of weights from the target/ground-truth point cloud to the source/generated cloud. Specifically, for the calculation of directional distances across both point clouds, we adhere to the relation $w(\mathbf{q},\mathbf{p}{\tiny 2,k})=w(\mathbf{q},\mathbf{p}{\tiny 1,k}),\ k=1,...,K$. The weights can be computed from $\mathbf{P}_1$ (resp. $\mathbf{P}_2$) and then shared with $\mathbf{P}_2$ (resp. $\mathbf{P}_1$), according to the task.
Our weight-sharing strategy is built upon the observation that if two point clouds are exactly the same, the weights obtained through the same calculation method should be equal for a typical reference point. If the weights are derived independently using the inverse distance for two point clouds, for the point cloud to be optimized/generated, the directional distance will be a high-order non-linear function of its points, thereby complicating the optimization process. The ablative study results in Table 7 of our manuscript show the superiority of the shared weights.

Comment 2. The generation of reference points are not explained very clearly.

Response: As explained in Section 3.2, the reference points serve to assess the disparity between the DDFs of the point clouds and are distributed in proximity to the implicit surfaces. Since each point in the point cloud is located on its underlying surface, we introduce offsets of Gaussian noise to displace these points away from the underlying surface while keeping them in close proximity. The standard deviations for the Gaussian noise are adjusted based on their distances to the nearest points in the point cloud, considering the non-uniformity of the point cloud. In the updated pdf file (i.e., the revised manuscript), we added one more figure, i.e., Fig. 3, to visually illustrate the generation process to help understanding.

Dear Reviewers

Thank you for dedicating your time and effort to reviewing our work and providing constructive comments and favorable recommendations. We have carefully considered and addressed all the concerns you raised in your review, as outlined in our response and reflected in the updated manuscript. As the Reviewer-Author discussion phase is nearing its conclusion, we eagerly await any further feedback from you. Should you have any additional questions, we would be delighted to provide detailed responses.