WIN: Variable-View Implicit LIDAR Upsampling Network

We sincerely thank Reviewer Gpfk for the valuable comments provided during this review. The point-to-point responses are as follows.

---

W1: The writting in Introduction is overpacking...the proposed method is not complex.

A: Thank you for your suggestion. Although our motivations and methods are simple, each of our modules is thoroughly considered and designed. First, we study past methods and point out the advantages and limitations of interpolation-based methods. We then visually show how to interpolate from different views (which, to our knowledge, has not been explored before) and point out the limitations of a single view. Considering the scene prior, we choose horizontal range view and vertical range view, and hope to fuse the two. In order to reasonably merge the results under the two views, we propose a selection module. Finally, in order to stably train the selection module, we formulate the loss function as needed (L287-290).

We further justify these designs. Regarding the fusion approach, we draw on other reviewers' comments and compare it with the weighted sum approach. The results show that the weighted sum does not perform as well as our chosen module. Regarding the loss function, we have demonstrated in ablation experiments that it not only correctly guides view selection (502-513), but also improves the stability of the algorithm (L514-525).

---

W2: Few state-of-the-art methods are compared in this manuscipt. And it is recommanded to add more comparision results.

A: Thanks for the comment, but we clarify that we compared all state-of-the-art scene-level point cloud upsampling methods. To dispel your doubts, we compare with recent object-level method, and the results are shown in Table 1. The results show that our method far outperforms [1] because [1] cannot be adapted to LiDAR data.

We also added the visualization results of depth completion in the Appendix A.2, which show that our method yields point clouds that are closer to the real geometry.

---

W3: Is it really essential to decompose LiDAR point cloud with HRV and VRV? Range view is enough.

A: The core point of our article is that the range view, although it contains all the information, can be limited in the interpolation framework**. We visualize the limitations of the range view in Figure 1. This limitation does not stem from a lack of information, but rather from the fact that the interpolated points are restricted to convex combinations of neighborhoods. This restriction provides a good prior in most scenarios, making the network easy to optimize. However, on non-smooth surfaces, interpolation from a single view may lead to theoretical infeasibility. For this reason, we propose to upsample from the horizontal distance view (HRV) and vertical distance view (VRV) separately, which not only helps to alleviate the limitation, but also naturally incorporates an understanding of the scene's geometric distribution: i.e., near-horizontal for the ground and near-vertical for objects and façades. All these factors make our approach go beyond existing methods and allow for a more accurate understanding of the geometric features in the scene.

If you are interested in the distribution of HRV and VRV, we visualize it in A.1, which intuitively shows the complementary of the two views, and gives direct evidence that using multiple views improves performance.

---

W4: Lacks of theoretical novelty.

A: We expected to explain our work in a more theoretical way too, but unfortunately the real world geometry is quite complex. At the same time, it seems too simple to prove that the interpolation range of multiple views is greater than that of a single view. Therefore, in the paper, we prefer to illustrate the motivation of our approach in a more intuitive way as well as through experimental results, cf. Fig. 1.

---

To summarize, we have designed an effective LiDAR upsampling framework based on an intuitive motivation. Our components are simple but carefully designed, and we justify our designs by extensive experiments. The theoretical supplement will be our next work. If you have any other doubts, feel free to point them out to us further.

Last but not least, we thank you again for your comments. We hope our answers solve your doubts.

[1] Qu W, Shao Y, Meng L, et al. A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsampling[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 20786-20795.

We sincerely thank Reviewer Lkn5 for the valuable comments provided during this review. The point-to-point responses are as follows.

---

W1: The authors should highlight their unique contributions and insights compared to these cross-view 3D works.

A: As you said, multi-view approaches have been explored a lot and they are impressive. But specifically, applying multiple views in the LiDAR upsampling framework is not easy, and requires more complex network design and fusion modules to achieve fine-grained feature extraction. We analyze the limitations of interpolation, go a step further to show the necessity of multiple views in the up-sampling task, and implement a new multi-view interpolation framework with a small number of parameters.

Previous methods usually extract features from multiple views and fuse them, but do not explicitly exploit the data distribution under multiple views. Different with previous work, Our method is essentially a new upsampler , which is independent with feature extraction.

---

W2 : In the Introduction, the contents of line-120 to line-127 and line-130 to line-138 are redundant with each other.

A: Thank you for your suggestions, we have revised the relevant parts in the updated paper.

---

W3: Would this module be further improved using the feature fusion strategy across different views? The concrete reason for selection rather than fusion applied in this work needs to be clarified.

A: Regarding the fusion method, our motivation is simple: we do not want to use the method of feature fusion and then predict range, because we have studied past work and found that even if we introduce several times more parameters and more complex structures like [1], it cannot superior simple interpolation methods. This is because the interpolation methods makes full use of known geometric information and has stronger generalization capabilities. In summary, we hope to retain the advantages of the interpolation algorithm.

After that, the key question is how to fuse the two predictions. Another reviewer Xvya had a related query, arguing that weighted averaging was more intuitive. However, in practice weighted averaging prevents both branches from being fully optimized and the final accuracy is not ideal. We show the comparison result on CARLA dataset in the table below.

Table1: Comparison of weighted sum and selection.

---

W4 : Some key works about the LiDAR point cloud interpolation-related tasks are missing.

A: Thanks for the reminder, although these methods are less relevant to our task, we have added references to these works in the updated paper.

---

W5: The experimental results are insufficient. For example, this paper lacks important qualitative evaluations of the ablation study and depth completion comparison.

A: We add the visualization results of depth completion in the A.2, which show that our method yields point clouds that are closer to the real geometry. We also provide visualizations of the complementarity of the two views to further justify our motivation in A.1. Our experiments provide a comprehensive comparison of related methods, which is sufficient to demonstrate the effectiveness of the proposed method.

---

W6: This paper is less polished and shows some poor presentations. For example, in line 210: r1, r2 is the upsampling factors; in line 279: use a MLP; in line 272: module(CSM), etc. Besides, the predicted view confidence g is missing in Figure 2.

A: Thank you for pointing out the typos in our manuscript. We have corrected them in the updated paper.

---

Q1: Can the proposed WIN help more downstream LiDAR-based works?

A: We add experiments on object detection and compare the performance of different methods in A.3. The results in the Table 2 show that our WIN can effectively improve the object detection result by upsample the low-resolution point clouds.

Table2: Results of object detection performance by different upsampling methods.

---

[1] Yang B, Pfreundschuh P, Siegwart R, et al. TULIP: Transformer for Upsampling of LiDAR Point Clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 15354-15364.

We sincerely thank Reviewer Xvya for the valuable comments provided during this review. The point-to-point responses are as follows.

---

W1: Why not extract features and predict interpolation weights in 3D space?

A: We emphasize that using range view is aiming to maintain the line characteristics of LiDAR. Previous work [1] has proven through complete comparative experiments that point-based methods cannot restore the scan line structure of LiDAR, and are significantly behind range view-based methods in various indicators. In order to further illustrate this problem, We compare the performance of the PUDM [2] and our method on the CARLA dataset (4 $\times$ upsampling) in Table 1. The experimental results show that it is much slower and has lower accuracy.

---

W2: Missing citations to recent object-level methods. Lack of camparison to object-based point cloud upsampling methods.

A: Thanks for your reminder, we have added citations to these works in the updated paper. However, due to the scanning characteristics of LiDAR, the acquired point clouds are larges-cale, uneven in density, and have line characteristics, which is very different from object-level point clouds. Many previous works [1, 3, 5-6] stated that range view-based methods are far superior to point-based methods, both in efficiency and accuracy. We briefly compared the recent object-level method PUDM [2] , and the results show that our method far outperforms [2] because [2] cannot be adapted to LiDAR data. The specific results are shown in the table below.

Table1: Comparison of object-level method [2] and our WIN.

---

W3: Why not apply a weighted sum to obtain the final prediction?

A: The motivation for using selection rather than weighted sum is twofold: 1) the two views have a clear geometric meaning, while the weighted sum does not, and 2) the weighted sum prevents the model from fully optimizing the two branches. We added a comparison experiment with the weighted sum method and the results show that our selection method is more advantageous as shown in the Table 2. Poor performance on IoU suggests that weighted sum methods may lose geometric information. We also added the visulization of the complementary of HRV and VRV in A.1 Fig

Table2: Comparison of weighted sum and selection.

---

W4: the upsampled point clouds matching the original point cloud resolution is unreasonable.

A: Since the KITTI data does not have a 128-line ground-truth, we can only provide a quantitative evaluation by downsampling and upsampling. Similarly, we can only employ this setup on downstream tasks to evaluate different models, as the current methods can not generalize to every input and output resolutions. We argue that our experiments are still meaningful, because improving the availability of sparse point clouds could facilitate the use of lightweight, cost-effective sensors in practical applications [4, 7, 10].

---

W5: The authors should compare object detection performance using the original KITTI point cloud data and the upsampled point clouds obtained with different methods.

A: As mentioned previously, we cannot apply all models to the original KITTI data. So we still upsample the 16-line point cloud into a 64-line point cloud and use the pre-trained model to provide object detection results. The results in the Table 3 show that our WIN can effectively improve the object detection result by upsample the low-resolution point clouds.

Table3: Results of object detection performance by different upsampling methods.

---

W6: [12] can achieve text-to-LiDAR generation. If a low-resolution range image is used as input to [12] with the text prompt "upsample 4x," how would the results from [12] compare to the method proposed in this paper?

A: Lidar upsampling task typically serve low-cost chips and real-time responsive autonomous driving systems that require models that are easy to deploy. Diffusion-based upsampling methods are all inadequate for this task because the sampling is too slow.

---

We sincerely thank Reviewer Qv6x for the valuable comments provided during this review. The point-to-point responses are as follows.

---

W1: (Motivation of Selection Module) The interpolation on the two orthogonal views is independent of each other and produces inconsistent results. Although the paper uses another module to select one of the points, it may not be a particularly plausible way of fusion.

A: The interpolated results of the two views are both geometrically intuitive, so most of the time the two results are both reasonable. And the key to the success of our method lies in the fact that we capture the widespread inconsistency between the two views.

Regarding the fusion method, our motivation is simple: we do not want to use the method of feature fusion and then regressing range, because we have studied past work and found that even if we introduce several times more parameters and more complex structures like [1], it cannot superior simple interpolation methods. This is because the interpolation methods makes full use of known geometric information and has stronger generalization capabilities. In summary, we hope to retain the advantages of the interpolation algorithm.

After that, the key question is how to fuse the two predictions. Another reviewer Xvya had a related query, arguing that weighted averaging was more intuitive. However, in practice weighted averaging prevents both branches from being fully optimized and the final accuracy is not ideal. We show the comparison result on CARLA dataset in the table below.

Table1: Comparison of weighted sum and selection.

---

W2: Interpolation using only a simple MLP may be difficult to process in different regions. Especially when the encoder features are only from the range view, the model may not possess the perception ability in HRV and VRV viewpoints.

A: Predicting interpolated weights is essentially predicting the similarity of a query point to neighboring points. ILN uses self-attention to learn relative relationships, but results in large matrix multiplications that make the memory footprint large. We use MLP and observe a very small loss in performance but significantly improving the training speed (40 min to 12 min every epoch), suggesting that correlations can be learned at a small cost despite large differences in features across regions.

---

Q1: Projection of HRV and VRV does not change its essence and still suffers from this problem.

A: Yes, we emphasize that no single projection can avoid the problem. Therefore our superiority essentially comes from the effective combination of multiple views.

---

Q2: how to solve the interpolation problem for blank areas, occlusions, or object edges?

A: It's a great question. In fact, the interpolation methods was originally designed to alleviate problems in those special areas [2]. We can discuss these cases separately, which demonstrate the advantages of the interpolation method:

• Empty area: Empty area is marked as 0 in range view. If the neighborhoods of a query ray are all blank points, in the upsampling perspective, we believe that the query ray must also be a blank point. Therefore arbitrary weights can be correctly interpolated in empty areas. In contrast, non-interpolating methods regress distances directly from features, often failing to regress 0 values unambiguously, which results in greater noise.
• Occlusion: In fact, the goal of upsampling is to add points to the observed surface, rather than considering the entire scene to be recovered. This is also the difference between upsampling and completion. Recovering occluded regions often requires strong prior information, closer to generative tasks. So occlusion is not a problem we need to solve, our goal is to generate high-resolution point clouds that respect low-resolution input.
• Object edges: Explicit methods based on regression have difficulty handling sharp edges, resulting in more noise, while methods based on interpolation can more easily avoid this.

---

Q3: The argument for predicting the best point through feature differences lacks a theoretical basis or a more in-depth explanation.

A: In the selection module part, we believe that conventional feature fusion methods are all feasible, such as addition, connection and attention. We use subtraction simply because 1) the $g$ we want to predict actually measures the difference between the two results, and 2) subtraction does not require any parameters. Other fusion methods can also be applied here, but this is not our focus.

---