CoFie: Learning Compact Neural Surface Representations with Coordinate Fields

Thank you for the detailed feedback, your acknowledgment of our theoretical proofs of fitting quadratic local patches, and the potential of our method. We will address your concerns as follows.

---

[W1, Baseline] Thank you for pointing us to NKSR. However, NKSR is not a directly comparable method to ours. As we mentioned in L189, L198, L247, and L249, our method works on the shape auto-decoding setting (the shape auto-decoder defined in DeepSDF), but NKSR works on shape auto-encoding. The differences are i) our method takes SDF samples as inputs, which is the same for DeepSDF and DeepLS, while the input of NKSR is the on-surface point cloud with normal as input; ii) After the shape decoder is trained, we fix it and perform back propagation to optimize the shape latent code, which is the same for DeepSDF, DeepLS and NGLOD, while NKSR uses feed-forward network to get the shape representation. Alternatively, as we mentioned in L248, even though the two settings are not directly comparable, we experimented with 3DS2VS, a SoTA shape auto-encoding method, as a reference method. To further address your concerns, we experiment with the mentioned NKSR method. We use 3K points as input without noise and use the full set of the 5 training ShapeNet categories for training; we test on the novel categories. We report the results on novel ShapeNet categories as follows. We note we also use the full set of 5 training ShapeNet categories for training 3DS2VS, while CoFie is trained on only 1000 shapes.

---

[W2, Scalability] Thanks for the valuable suggestions. We agree that CoFie can be extended to scene-level shapes as it uses local shapes. However, most of the scene meshes are not watertight, where we cannot use the point sampling method of DeepSDF to get the data. DeepLS paper provided a method to fake the input by using on-surface points and surface normal. However, their scene-level code is not released. We are unable to re-implement the scene-level data sampling method during the short rebuttal period, and we are committed to adding the result in the next version.

---

[W3.1, Generalization claim] We thank you for the detailed feedback on generalization. However, the generalization in the NGLOD paper (reported in their Table 4) has a different definition from ours, even though we use the same terminology. As discussed at the beginning of Sec 4.3 of the NGLOD paper, their generalization works on the setting of i) training a model on a single shape, ii) applying the trained model to novel shapes, according to their description of “We now show that our surface extraction mechanism can generalize to multiple shapes, even from being trained on a single shape.” However, the generalization defined in our setting is i) training on a set of shapes, ii) applying the trained model to novel shapes, which is the setting in DeepSDF, DeepLS, and our work. We alternatively note that NGLOD doesn’t support training on multiple shapes. To further address your concern, we report the performance of NGLOD on Thingi by training the MLP on a ShapeNet Chair (denoted as NGLOD pre-trained). The performance is significantly worse than ours according to Table 3 in our paper. The reason is that the local shape distribution of the training shape and testing shape are very different. In contrast, our model can be trained on multiple shapes (1000 shapes with 3 million local surfaces), which enables us to learn general shape priors.

---

[W3.2, Generalization across density] We report the performance of our work using only 10% SDF samples as input. We observe that the performance of CoFie is robust to the number of SDF samples.

---

[W4, Proof] Eq. 14 essentially applies Taylor expansion in x, y, z and ignores third-and-higher order terms in x, y, and z. Please see details in the rebuttal pdf.

---

[W5, Reference] Thanks for the detailed feedback. The missing references at the mentioned line are [22, 38, 8, 34], which are the same with the ones we discussed in the second paragraph of the introduction. We will add them in the revision.

---

[W6, Shape at voxel boundary] Good question! As mentioned in the paragraph of L211 of the paper, we deal with the problem by using an expanded “receptive field” for each voxel. In practice, we model a local shape using the SDF samples belonging to the neighbor voxel using a distance of 1.5V, where V is the size of the voxel. This improves the continuity of the shape at local voxel boundaries.

Thank you for your detailed and positive comments. Thank you for acknowledging the novelty, the theoretical foundation, and the good performance of our work. We will address your comments as follows.

---

[W1, Baselines] Although we understand your concern, we cannot find more directly comparable works. For example, GIFS, HSDF, and UODFs work on modeling non-watertight surfaces based on UDF. The input of GIFS and HSDF is the point cloud, which follows the setting of OccNet. However, the input of our method is SDF samples, where the points can be not on-surface. This follows the setting in DeepSDF, DeepLS and NGLOD. The mentioned UODFs work is a shape-specific method, which is directly comparable to NGLOD, but both NGLOD and UODFs are different from our setting of generalizable shape representation as we noted in L244. To further address your concerns, we report the performance of UODFs as follows on the Thingi mesh ID 252119. In general, UODFs demonstrate better performance than NGLOD. Our method, which takes much less time for inference and is not shape-specific, demonstrates comparable results.

---

[W2, Compactness] Thanks for your suggestion. Except for the number of parameters, our method also demonstrates a faster convergence speed over the directly comparable baseline DeepLS. When using the latent code size of 125, our method achieves comparable performance with DeepLS using only 61% training iterations. During inference, we set the same number of iterations for our method and DeepLS, where the inference time is roughly the same.

---

[W3, Code release] We are firm supporters of open source. We commit to releasing the code right after the decision results.

---

[W4, Minors] Thanks for the detailed feedback. We will incorporate them in the revision.

---

[Q1, Computational cost] The quadratic layer makes the training/inference speed 14% slower. The used GPU memory increases by 3%.

---

[Q2, Grid size] We experiment with a grid size of 16x16x16. The model shows a 4.96 chamfer distance error (according to Table 1 in the paper). We agree that the grid size influences the performance. In the meantime, we want to clarify that using a large grid resolution doesn’t increase the complexity of our model significantly. Our method works on valid voxels that intersect with the shape surfaces, which represent only 6% of all voxels when using a resolution of 32x32x32. This property makes our method efficient. When extending to a resolution of 64x64x64, only 1.2% of voxels are valid, where the complexity is only 1.6 times of resolution 32 and is far less than a cubical factor of 8.

---

[Q3, Aggregation function] This function is a general representation of blending the local shapes into the global geometry. Using the indicator function is a special case of it, and is common in implementations, including DeepLS. Our method can also be extended to other blending methods, however, the blending method is not the focus and contribution of this paper.

---

[Q4, Hyper-parameter] In our experiment, we also tested using 2 quadratic layers and the experiment showed a 2.20 error, which is larger than using one layer (according to Table 1 in the paper).

Thanks for your positive comments, and your acknowledgment that our theoretical proof is sufficient and the articulation is clear. We will address your comments as follows.

---

[W1, grid size] Our method has demonstrated good performance in modeling detailed geometry as shown in Figure 3, Figure 6, and Figure 7, including thin structures, non-smooth local surfaces, as well as edges and corners. Alternatively, we note that our method can be easily extended to high resolution without increasing the complexity a lot. Our method works on valid voxels that intersect with the shape surfaces, which represent only 6% of all voxels when using a resolution of 32x32x32. This property makes our method efficient. When extending to a resolution of 64x64x64, only 1.2% of voxels are valid, where the complexity is only 1.6 times of resolution 32 and is far less than a cubical factor of 8.

---

[W2, Representing color] Since we follow the common setting in implicit representation works [4,24,30,31,29,3,27,49,37], representing color is currently out of the scope of our work. Moreover, as shown in Figure 3, Figure 6, and Figure 7, our method can represent the local shape boundary between voxels accurately. This is strong evidence that it can also work for color. Moreover, as mentioned in L211, we improve the smoothness of the surface across different voxels by expanding the "receptive filed" if each voxel. This can also contribute to better color modeling capability.

---

[W3, Representing rich curves] We kindly note that the reported number on Thingi is from the model trained on ShapeNet. Thus, its comparable performance with the shape-specific NGLOD demonstrates the power of CoFie on complex shapes.

---

[W4, Reference] Thanks for the detailed feedback. We will revise the reference.

---

[Q1, Training set] Training on 1000 shapes of ShapeNet is a common setting in shape auto-decoding works. We strictly follow the setting in DeepLS for using 1000 shapes. Please see more details in the experimental setup section of the DeepLS paper. Besides, we want to clarify that the 1000 shapes contain 3 million local surface samples, which is sufficient for training. This property demonstrates the data-efficient nature of our work.

[W5, Evaluation] You mentioned that DeepSDF overfits single shapes, but in fact, it trains the MLP decoder on a set of training shapes, and optimizes the latent code for each shape during testing. The MLP decoder is shared for all shapes, thus, it is not overfitting. Please see details in the DeepSDF paper and the training split of the DeepSDF official GitHub repo. We strictly follow this setting. We agree that NGLOD overfits each single shape (shape-specific) for both the MLP decoder and shape representation. As we discussed in L245, shape-specific methods generally have better accuracy as it is specialized in a single shape. We use it as an “oracle” comparison, and the results in Table 3 show the strong performance of our method.

---

[W6, Quadratic MLP] The use of quadratic MLP is based on our analysis with a strong theoretical foundation (Sec 3).

---

[W7, Paper details] The clarity of our writing is acknowledged by other reviewers, including “proofs are sufficient and articulation is clear” (4hsy) and “well illustrated and easy to follow” (Kq6M). As we mentioned in L201, the inputs are SDF samples, not point cloud or mesh. The density and method of SDF sampling strictly follow DeepSDF, which is also the same in our experiment of DeepLS and NGLOD. For 3DS2VS, we use 10K points as input for a fair comparison, as DeepSDF sampling has the number of SDF samples with this quantity.

---

[W8, Statements] Shape latent code is the official terminology in 3DS2VS. See the second line of the caption of Figure 2 in 3DS2VS. “Latent code” doesn’t mean there is only one code. We searched in the 3DS2VS paper, and there is not a single matching regarding the term “latent code clouds” you mentioned.

---

[W9, Computation] Our method is computationally efficient. The 1000 training shapes contain 3 million local patches, and we train on the patch level. When using the latent code size of 125, our method achieves comparable performance with DeepLS using only 61% training iterations. This is strong evidence that our method is computationally efficient.

---

[W10, Reference] We are happy to cite and discuss the mentioned papers in the revised version of our paper.

---

[Q1, Input] As we mentioned above, we are performing shape auto-decoding (updating latent code via back propagation and optimization during testing), where the standard input is SDF samples. The input of DeepSDF, DeepLS, and CoFie are strictly the same. We don’t have an encoder as we perform shape auto-decoding. Regarding generalization, there is no correlation between generalization and whether it uses back propagation. See DeepLS, which is a generalizable method using back propagation, for more details.

---

[Q2, Baselines] As we mentioned above, DeepSDF, DeepLS, NGLOD, and our method use strictly the same inputs. We follow the 3DS2VS official code for training and testing.

---

[Q3, Training data size] Again, we are doing shape auto-decoding, which is a different task from auto-encoding. We agree that shape auto-encoding requires a large amount of training data as it learns priors to map the shape/point cloud into latent code. However, this is not true for shape auto-decoding as these methods optimize the latent code during inference. For example, DeepLS also uses 1000 shapes for training (as discussed in the experiment setup paragraph in the DeepLS paper) and we follow their setting strictly. Our method works on a patch level, and the 1000 shapes introduce 3 million local patches, which are sufficient for training.

---

[Q4, Using Fourier embedding] Good question! We experiment with using Fourier embeddings and show the results below. To make the comparison fair, we use a code size of 64 and Fourier embedding with a dimension of 64, which makes the input dimension the same as using point coordinates (3 dimensions) with a 125-dimensional latent code. We provide the results as follows, and they show the effectiveness of our method. Alternatively, we note that our idea of coordinate field is orthogonal to using positional encoding, as we can also apply positional encoding on the transformed coordinate system.

---

[L1, Representing complex shapes] We have already shown the capability of CoFie in Figure 3, Figure 6, and Figure 7, which include topologically challenging shapes like thin structures and unsmooth local surfaces, as well as edges and corners.

---

[L2, Computation] Please see W2 and W9. Alternatively, we report that NGLOD takes 105 minutes for a single shape, which is much slower than ours.

Thanks for your detailed feedback, and your acknowledgement of the efficiency of our model, the clarity of our writing and figure, and our clear ablation study. We will address your comments and some factual errors in your comments as follows.

---

[W1, AutoSDF and SDFusion] The mentioned papers are significantly different from our work. First, AutoSDF and SDFusion have a different target from ours. In detail, AutoSDF and SDFusion target to learn the latent space of SDF represented shape, which enables them to perform downstream applications of shapes that belong to specific categories, e.g., chairs and airplanes. In contrast, our work focuses on shape representation, where we prove that using coordinate fields with better MLP designs can represent the shape of general categories better with both experimental and theoretical evidence. Moreover, AutoSDF and SDFusion use shape auto-encoding to learn the latent space. Differently, our paper works on shape auto-decoding (as mentioned in L189 and L243), or termed as auto-decoder in DeepSDF and DeepLS paper. As discussed in L248, shape auto-encoding works can not be directly compared with our method. At the same time, we agree with your comment that we both use grid-based SDF. However, this is very common and standard in related research and we have discussed them in Sec.2, including [4,21,36,19,32,35,30]. We never claim that using the grid-based SDF is one of our novelties. We are happy to add the discussion of the mentioned paper in the revision for more clarity.

---

[W2, Complexity] Our method doesn’t scale with cubical complexity. As illustrated in Fig 2, our representation only focuses on grids that contain the shape surface. Thus, we actually use sparse voxels, and those voxels that don't contain surfaces are discarded. When using the resolution of 32x32x32, only 6% percent of the grids are valid. When using the resolution of 64x64x64, only 1.2% of voxels are valid, where the complexity is only 1.6 times of resolution 32 and is far less than a cubical factor of 8. This property is similar to DeepLS.

---

[W3, Evaluation] The effectiveness of our evaluation is acknowledged by other reviewers, including “convincing evaluation” (Kq6M), “good ablation study” (LpYZ), “Experimental results demonstrate strong generalization capability of CoFie, outperforming previous methods in terms of accuracy and efficiency”, and “the results on two datasets show great potential” (58nR).

To further address your comments, first, we would kindly remind you that CoFie works on the shape auto-decoding task (optimization-based) as we mentioned in L189, L198, L247, and L249. CoFie doesn’t work on the auto-encoding task (feed-forward-based) you mentioned. Second, regarding our claim of “represent arbitrary shape”, we have verified the claim in our experiment in Sec 5, and it is appreciated by other reviewers, including “strong generalization capability” (4hsy), “outperform previous generalizable methods” (Kq6M), and “show great performance on both synthetic and real data” (58nR). As we mentioned in L277, we train the model on 5 ShapeNet categories, and we test the model on 10 novel ShapeNet categories and also novel real objects from novel categories. Results in Table 2 and Table 3 are comparable to Table 1, demonstrating the strong generalization capability. Conceptually, our method doesn’t have any category-specific designs, and the local patch-based method is naturally generalizable to any shape [4].

Finally, we would like to address your concerns regarding the dataset size. First, using a thousand instances for training is typical in previous works [24,4]. We strictly follow the setting in DeepLS [4]. Second, to further address your concern, we report the result of 3DShape2Vec using the full set of the 5 categories in Table 1 below. We alternatively mention that 3DS2VS is a shape auto-encoding method and it is not directly comparable to shape auto-decoding methods including DeepSDF, DeepLS, NGLOD, and our method. We have mentioned this in L248. Shape auto-decoding generally has a much better performance than shape auto-encoding. With more data, the shape representation accuracy of 3DS2VS is still not as good as shape auto-decoding methods, especially in novel categories.

---

[W4, baselines] We respectfully disagree with your points. You mentioned that DeepSDF uses a PointNet encoder, but actually there is no encoder in DeepSDF as it performs shape auto-decoding (or termed as auto-decoder). By searching “PointNet” in the DeepSDF paper, we don’t find any matches in the method section. Please let us know if there is any following work that modifies DeepSDF with a PointNet encoder. This is also true for DeepLS. The mentioned 3DILG paper works on shape auto-encoding, and actually the SoTA of shape auto-encoding is 3DS2VS and the performance of 3DS2VS is reported in Table 3 of 3DS2VS paper, which outperforms 3DILG. Thus, we didn’t compare with 3DILG. Again, we want to emphasize that shape auto-decoding methods and auto-encoding methods are not directly comparable. We would be happy to compare with more shape auto-decoding methods if we missed any.

---

We will continue to address the question in the official comment reply.

We searched in the 3DS2VS paper, and there is not a single matching regarding the term “latent code clouds” you mentioned. Shape latent code is the official terminology in 3DS2VS.

I refer authors to the 3DShape2VecSet paper introduction and contribution 1 in particular: "We propose a new representation for 3D shapes. Any shape can be represented by a fixed-length array of latents". I also refer authors to section 5.1 to 3DS2VS paper that explicitly states: "In our final model, the number of latents 𝑀 is set as 512, and the number of channels 𝐶 is 512". Finally, I refer authors to the source code of 3DS2VS (LL182-283).

“Latent code” doesn’t mean there is only one code

I don't believe authors are making this argument in good faith. DeepSDF paper explicitly states that shape is represented by 1xD vector. 3DS2VS paper explicitly states that shape is represented by 512xD tensor (see above). In scientific research, precision of language matters. For example, nobody call 'matrix' (2D tensor) a 'vector' (1D or 0D tensor).

The use of quadratic MLP is based on our analysis with a strong theoretical foundation (Sec 3)

The argument in Sec. 3 is done in comparison to 'shallow MLP using ReLU'. This assumption does not hold in majority of practical cases: majority of methods either use non-shallow MLPs, or non-locally linear activations like (ELU, GEGLU, GELU, SiLU, etc), or both.

We experiment with using Fourier embeddings and show the results below <...>

Thank you! This is solid empirical evidence in favor of proposed method and I recommend including it in updated version of the paper.

Our method works on a patch level, and the 1000 shapes introduce 3 million local patches, which are sufficient for training.

I respectfully disagree. The fact that models sees 3 mln local patches of 1000 ShapeNet shapes during training does not mean that it will be able to generalize to 50000 unseen ShapeNet shapes. I also want to note that 3DILG also utilizes local surface point patches (512 per shape) that are randomly sampled for each iteration. It means that for each iteration of 1000 shapes 3DILG sees ~0.5 mln of unique local patches, so this property is not unique to the proposed method.

Ability to deal with topologically challenging shapes

I respectfully disagree. Maybe the issue here is that we that we consider different types of shapes topologically challenging. I refer authors to Fig. 8 of 3DS2VS paper for examples of topologically challenging shapes that I find topologically challenging (esp. rows 2,3,5,8,9). My concern is that proposed method won't be able to handle such shapes (and 3DS2VS is capable of it).

Our method is computationally efficient <...>.

I respectfully disagree. While proposed method might be more computationally efficient than DeepLS (2020 paper), it still requires 4 24GB GPUs and 24 hours to train on 1000 shapes. To me, it means that it will take at least a month to train this model on full ShapeNet. But given more pressing issues with evaluation and baselines, this does not hold high weight in my evaluation.

You mentioned that DeepSDF overfits single shapes, but in fact, it trains the MLP decoder on a set of training shapes, and optimizes the latent code for each shape during testing.

My confusion is understandable since proposed method compares both to NGLOD that was trained in overfitting setting and 3DS2VS that is trained in auto-encoding setting. That being said, I apologize for the confusion but want to point out that this confusion does not invalidate my concerns.

The clarity of our writing is acknowledged by other reviewers <...>

The goal of review process is to gather unbiased feedback from fellow researchers, so it is only natural that opinions differ.

I appreciate very detailed feedback by the authors: it clarified a lot of details in the original submission. Below I will address main rebuttal points (and I will address other rebuttal points in separate comments).

After the rebuttal, I have no doubts that Cofie proposes a novel method that can be viewed as a sparse irregular latent cloud/grid/array of 1/32 sized voxel cells that intersect with shape surface. These cells are decoded via novel quadratic MLP decoder (and this choice is ablated in the rebuttal). This makes it different from 3DILG, Dynamic Code Clouds, 3DS2VS, AutoSDF and SDFusion.

Unfortunately, my three major concerns about evaluation are not addressed in the rebuttal.

Concern 1. Choice of baselines for auto-decoding setting (Tab. 1) is very weak.

DeepSDF is the 2019 paper. DeepLS is a 2020 paper. Since then, a lot of strong shape representations were proposed: Local Implicit Grids (2020), AutoSDF (2022), SDFusion (2023), Dynamic Code Clouds (2023), 3DILG (2022). I don’t expect authors to compare to all of them but in addition to 3DS2VS proposed method needs comparison with 2-3 recent methods that utilize latent grids/clouds/arrays.

Concern 2. Proposed method is evaluated via training on 1000 ShapeNet shapes (Table 1).

Authors claim that “using a thousand instances for training is typical in previous works [24,4]”. Again, DeepSDF [24] is the 2019 paper; DeepLS [4] is the 2020 paper. For comparison, 3DS2VS trains on 48597 ShapeNet shapes (auto-encoding task) and evaluates on 2592 shapes. In fact, recent works use at least ShapeNet-13 (see Dynamic Code Clouds papers). In my opinion, the proposed method must either be trained and evaluated at least on ShapeNet-13 (as Dynamic Code Clouds does) or results for 1000 shapes must be averaged across at least 5 different runs on different 1000-shape subsamples of ShapeNet.

Concern 3. The paper only uses Chamfer Distance as an evaluation metric.

Majority of baselines use at least 2 evaluation metrics. DeepSDF [4] uses CD and EMD; 3DS2VS uses CD, IOU and F-score. Only DeepLS uses just a CD. Given the strong claims in the paper and limited training/test size in the submission, I recommend having at least one additional evaluation metric.

If the paper with current evaluation pipeline is accepted, my concern that it might create bad precedent for follow-up work that is going to use the same evaluation pipeline. Thus, my rating remains unchanged.

======================================

Below are additional main rebuttal points I want do address:

By searching “PointNet” in the DeepSDF paper, we don’t find any matches in the method section

Authors are right and I acknowledge my mistake and apologize for the confusion. But I want to point out that this mistake does not invalidate my concerns outlined above and the fact that these concerns were not sufficiently addressed in the rebuttal.

The effectiveness of our evaluation is acknowledged by other reviewers

The goal of the review process is to gather unbiased and diverse feedback from fellow researchers, so it is only natural that opinions differ.

Performance of CoFie on Thingi, comparison with NGLOD/UODFs

We would like to clarify NGLOD/UODFs are shape-specific auto-decoding methods. Different from generalizable shape auto-decoding where only the latent shape representation is optimized, NGLOD/UODFs also optimize the model for each specific shape. Thus, shape-specific auto-decoding methods naturally have better performance. We include the comparison of the two settings as follows.

The close performance between CoFie and NGLOD/UODFs in Table 2R demonstrates the effectiveness of our work. We also report the performance of CoFie trained by 20K shapes as follows based on Table R2, where CoFie is further improved with more training data.

Similarly, we would like to clarify the two kinds of methods (GAD v.s. SSAD) cannot be directly compared. This analysis is performed as it was required by the reviewer for understanding CoFie’s performance.

---

We thank the reviewer for the suggestions on understanding the performance of CoFie in the context of recent generalizable shape auto-encoding and shape-specific auto-decoding works. Although we don't agree the methods of different settings are directly comparable, we hope the analysis could provide better understanding of our model. We also hope the added results and analysis could clarify the effectiveness of CoFie in the context of using more data for training. We appreciate the time and effort spent in the discussion period, and we will incorporate the discussion content and all your suggestions in the revision.

Thanks for the feedback. We would like to conclude the discuss as follows.

Tables 1-2 provide comparison with either very old models (DeepSDF, 2019; DeepLS, 2020) or with models that were not designed for GAD setting(3DS2VS).

• We agree that DeepSDF and DeepLS are old methods. In the meantime, we kindly remind that they are the only two directly comparable methods for GAD as far as we know. We are happy to compare with any baseline for GAD if you could provide any references. Whether DeepSDF and DeepLS are old is orthogonal to our contributions in this work, as our contributions are ablated.
• We note we have mentioned many times GAE and SSAD methods are used as reference and they are not directly comparable methods. See Line 243 for categorizing generalizable and shape-specific models. In Line 247 we clarify the setting of generalizable shape auto-encoding and noted it is not directly comparable. We also note classify each method, as in Line 251, 253, 255, 258.
• The reviewer got an wrong impression by him/herself and urged us comparing with GAE methods because him/herself works on GAE (mentioned as "as a person who trained reconstruction and generative models on full ShapeNet", where "reconstruction" is GAE) and this leads to the biased reviews towards GAE. Again, we never claim we should compare GAD methods to GAE and SSAD methods. Instead, this is the understanding of the reviewer. Please see the initial review of V5fn for the details of requesting GAE method experiments.

Even if we consider GAD setting, models are evaluated based on training on 1000 shapes.

• We acknowledge that we trained the model with 1000 shapes.
• Using 1000 shapes is standard in prior works. See DeepLS.
• At the same time, we would like to point the review to the results provided during the rebuttal period of training CoFie on 20K shapes.
• The new experiment demonstrated that CoFie shows good performance in the context for both low-data and larger-data regime.

Choice of baselines in Table 3 has the same issue as choice of baselines in Tables 1-2.

• This is your own fault for comparing these methods together. Again, we note we have mentioned many times GAE and SSAD methods are used as reference and they are not directly comparable methods. See ine 243 for categorizing generalizable and shape-specific models. In Line 247 we clarify the setting of generalizable shape auto-encoding and noted it is not directly comparable. We also note classify each method, as in Line 251, 253, 255, 258.
• The reviewer didn't realize the differences between GAE, GAD, SSAD until the last reply. Even after he/she realized this, he/she still attacked us using his/her own wrong understanding.
• Again, at the very beginning, we note we have mentioned many times GAE and SSAD methods are used as reference and they are not directly comparable methods. See ine 243 for categorizing generalizable and shape-specific models. In Line 247 we clarify the setting of generalizable shape auto-encoding and noted it is not directly comparable. We also note classify each method, as in Line 251, 253, 255, 258.

We thank you for the positive comments and your acknowledgment of the thoroughness of our ablation study. We will address your comments as follows.

---

[W2, Benefit of Coordinate Field] We kindly note that the coordinate field is a part of our representation, as the coordinate field is involved in the computation of decoding shape surfaces. It makes the learning more efficient, improving the geometry modeling capability, and helping us learn the MLP in a more compressed local patch geometry space.

---

[Q1, Coordinate field] A field is defined as follows in Wikipedia: “In science, a field is a physical quantity, represented by a scalar, vector, or tensor, that has a value for each point in space and time.” Our coordinate field is a discrete field, where the coordinate frame value of a 3D point is associated with the local patch it belongs to. We note that this discretization is the key to defining local patches and enabling the geometry-aware coordinate field and its initialization.

---

[Q2, Regularization] Thanks for pointing this out. We use the same regularization as previous works that penalize the latent code values, following the standard regularization term in DeepSDF and DeepLS. We didn’t include the term in the loss function as it is standard. We will revise the text for clarity.

---

[Q3, Well-oriented normal] Great question! We clarify that at test time, we don’t have full SDF but SDF point samples (following DeepSDF, DeepLS, NGLOD, etc), and the estimated normal can be imperfect. Thus, as we mentioned in equation 9, we optimize the latent code and the coordinate frame jointly, which makes our model more robust to noise in the estimated normals.

---

[Q4, Supervision at test time] During testing, we follow practices from DeepSDF, DeepLS, and NGLOD. We sample points (with known SDF values), including points close to the surface, which is a standard point sampling method for shape auto-decoding. Thus, the points are not on-surface points, but also we don’t have access to the full SDF (we only have point samples of the SDF). In evaluation, we compare with DeepSDF, DeepLS, and NGLOD inference on the same data with our method.

---

[Q5, Inference time] We achieve an inference time of 10 minutes in the setting of optimizing a single shape, which is much faster than the baseline NGLOD (105 min). Moreover, our method benefits from the capability of inference on a batch of shapes by sampling batched local shapes from all of them. This property further improves the efficiency of our method. NGLOD does not support batched inference since it optimizes a non-regular grid which is different for each test shape.

---

[Q6, L243] Thank you for catching that! We meant three types of methods: 1) generalizable shape auto-decoding methods, 2) shape-specific shape auto-decoding methods, and 3) generalizable shape auto-encoding methods, which are used as a reference and are not directly comparable (as discussed in L247). We will clarify this in the revision.

---

[Q7, missing references] Thanks for the detailed feedback. These references include previously mentioned works including [22,38,8,34,30]. We will revise the references in revision.