Level-Set Parameters: Novel Data for 3D Shape Analysis

I am a little concerned about the novelty and the meaning of doing so.

First, pose-dependent shape generation seems to be well-explored, not only for objects but also for human faces/bodies. I believe a survey of these works is necessary. Also, what are the novelties for this work given existence of pose-dependent generative networks?

Second, the authors may argue that the pose network can be clearly handled separately. However, with more and more shapes, can the network really handle them separately? For example, given fixed shape parameters and random pose input, it is hard to justify that the network will produce shapes with variance only in poses rather than shapes. Even if it is explicitly enforced during training, can the network inherit this capability during inference (especially when data scale become large).

Third, as a decoder-only network, it seems that there is no possibility for it to handle novel shapes, which should be considered a big limitation. Therefore, it is hard for me to appreciate its application value.

1 Need to construct level-set parameters for each shape before shape analysis.

2 Although the performance is improved, I feel that the proposed data modality complicates the task. Take the shape classification task as an example: traditional methods can directly take conventional data as input. However, with the proposed method, a dataset construction step is required. This involves using a Hypernetwork to generate the weights of another neural network (SDF), which is trained on traditional data modalities such as points and normals. Then, a specialized neural network (semantic learning network) needs to be designed to handle the large dimensionality of parameters as input for classification training. This reduces the flexibility in network design choices. Also, if users want to add new data or a new shape, they may need to retrain the Hypernetwork and SDF network. If so, this is time-consuming.

3 The supervised methods in Table 2 are just PointNet and PointNet++. Although these methods are famous, they are also old methods. Please consider including more recent works.

4 Table 2 should also consider reporting the number of parameters of each model for a fair comparison.

5 PaRI-Conv's performance is as competitive as the proposed method in Tables 2 and 3. PaRI-Conv even outperforms the proposed method in some categories in Table 2.

6 I do not like this statement "In addition, we observe that our classification network converges rapidly within a few epochs, consistent with the findings in (Dupontetal.,2022)." The time of dataset construction should be taken into account.

7 I feel it is too ambitious to propose a new data modality. If the focus of the paper is on a new data modality, I would expect more versatile shape analysis beyond just classification, pose estimation, and retrieval. Additionally, I would expect the modality to offer more flexibility for neural network development rather than requiring a specific encoder design. Furthermore, I would expect more groundbreaking improvements, but the enhancements in classification and retrieval are not significant.

Clarity and Presentation: The paper could benefit from improved clarity in communicating its main ideas. For example, the rationale behind certain design choices is not well-articulated, making it challenging to understand the intuition behind the approach. Specifically, Section 3.2 & 4.1 would benefit from a revision to enhance readability, as it is difficult to get an intuitive understanding of what is going on in these sections. Below are few of the points which need improvements:

• In section 3.2, why a 2 stage approach for predicting the parameter decomposition is used?
• Why trainable parameters are initialized as zero in line 236?
• What is the intuition behind Equation (6)?
• Many things in section 4.1 seems hard-coded. For e.g. What is the intuition behind concatenating the weights and biases of the first SDF layer? What is the intuition behind using parameters of only layer 2 and 7 of the SDF network?

Because of these points I feel section 3.2 and 4.1 needs a major rewrite.

Results and Evaluation: The experimental results are presented on relatively simple datasets, such as ModelNet40 and ShapeNet. It would be valuable to evaluate the method on more complex, large-scale datasets like Objaverse to better assess its robustness and scalability. Will this method work in more modern datasets? Some quantitative or qualitative evaluation on these dataset is helpful. Given the relatively small improvements in quantitative comparison in Table 2 and 4, I have doubts about the robustness of this approach. And hence, a more extensive evaluation on complex datasets like Objaverse is required.

• The motivation of this submission is vague. The experimental section demonstrate tasks mostly belonging to point cloud processing, the motivation of leveraging SDF formulation is not justified.

• The exposition in Sec. 3.1 is poor, there is no motivation (or at least reference to existing works) on the formulation of hyper-network. Why is matrix-form entry necessary? How is it related to better encoding of rigid transforms?

• Similarly, Sec. 4.1 threw a pile of architecture details on the readers without any reasoning. I found it hard to get why the network leads to semantic understanding of shapes.

• The running time is not fully reported. It is only mentioned in the end of Sec. 5 that it takes 50 seconds for registration, which is clearly slow.