PIN: Prolate Spheroidal Wave Function-based Implicit Neural Representations

First and foremost, the authors would like to thank the reviewer for their thoughtful and insightful questions, which have provided us with an excellent opportunity to clarify and strengthen our work.

W1:

We sincerely thank the reviewer for their insightful question. We acknowledge that the numerical implementation has not been detailed in the main text and will make sure to incorporate this in the revised version. We would also like to gently point out that a more comprehensive description is available in Appendix regarding the approximation method we used. Specifically, we utilized cubic spline approximation, which provides a third-order approximation between two successive points while maintaining continuity and differentiability. A simple regression would often fail to retain the exact data point, and the differentiability properties between discrete points, where differentiability properties are indeed needed during the backpropagation. Therefore, the cubic spline approximation, which is of third-order and computationally efficient, serves as an optimal choice, and does the intended task.

W2: We are thankful for the reviewer regarding his question. The following table summarizes the occupancy field results for different INRs.

Table: IoU for Occupancy Fields

The following table summarizes NeRF results

Table: NeRF for different objects

W3: We are thankful for the suggestion by the reviewer. The authors will make sure to improve the presentation of the paper by utilizing vector graphics and the suggested commands in the revised manuscript.

Q1: We utilized a cubic spline approximator, which is order 3. The authors do not believe this would affect any theoretical properties of the PSWFs, as when we get the discretized solution to the governing equation for PSWF, we utilized a cubic spline approximator between every successive points. This cubic spline approximation is necessary because a continuous function, rather than a set of discrete points, is required as an activation function in a neural network. So as far as our knowledge is concerned there is no any potential way of diminishing any theoretical properties. However, if a simpler approximator such as a basic quadratic or cubic polynomial, or even a neural network, were used to approximate the discretized solution, it could potentially compromise some of the properties of PSWFs. These methods do not guarantee passing through the discretized points, which could lead to inaccuracies in differentiation and, consequently, incorrect outcomes during backpropagation.

Q2:

We are grateful for reviewer regarding this question. As explained in Section 6 of the paper, we utilize an explicit control mechanism, where the frequency is now governed by the parameter $\omega$. This makes $\omega$ the frequency-controlling variable instead of $c$. Now, regarding how to effectively select the parameter $\omega$, one possible approach is to perform a grid search to identify the values that maximize the results. However, as you mentioned, this often leads to suboptimal outcomes when a different dataset is presented, particularly in NeRF and other applications where strong generalization is crucial. (Incidentally, this is the standard approach adopted by many INR baselines.) Instead of relying on a grid search, we use the most straightforward configuration for $\omega$, which is $\omega = 1$. We initialize all experiments with this value and make $\omega$ a learnable parameter. Consequently, it gets adjusted dynamically based on the loss function of the intended task. This approach benefits from our spline approximation, which allows the activation function parameters to traverse in the loss landscape more effectively, as explained in the Appendix of the paper

First and foremost, the authors would like to thank the reviewer for their thoughtful and insightful questions, which have provided us with an excellent opportunity to clarify and strengthen our work.

W1:

For all the experiments, we have ensured the optimal parameters of the other methods have been used. However, for the proposed method, every experiment has been conducted with the same parameters unlike others. As can be seen, these baselines do require specific fine-tuning to get the results. However, PIN does not require any of those conditions.
The following table summarizes the activation function parameters utilized for each application.

Table: Configurations for WIRE, SIREN, GAUSS, and PIN Across Experiments

W2:

We are thankful for the reviewer regarding the question. The following table summarizes the training speed corresponding to different INRs. As can be seen from this table, PIN's runtime is comparable with that of previous INRs.

Table: Convergence Time Across Methods

W3:

We are thankful to the reviewer for raising this question. We attempted the image super-resolution task on the "Boy" image from the Set14 dataset [ref] and the "Cameraman" image. The following table summarize the results for the "Boy" and "Cameraman" images in 2nd and 3rd columns respectively.

Table: PSNR for Image Super Resolution

[ref]. Awesome-Super-Resolution/dataset.md at master ·559 ChaofWang/Awesome-Super-Resolution — github.com.560 https : / / github . com / ChaofWang / Awesome -561 Super-Resolution/blob/master/dataset.md.562 [Accessed 20-11-2024]

Q1:

We are thankful for the question, and the suggestions. The authors acknowledge that PIN is compared with initial methods. When considering improvements to [1], we believe there is potential to enhance its methods using PSWFs. However, the core idea of [1] is to employ a fixed Fourier basis and decompose the neural network weight matrix into a product of trainable and fixed Fourier basis matrices. Given this, a key question arises: how can the Fourier basis and PSWFs be effectively combined? A possible approach is to modify [1] by incorporating Fourier basis elements of PSWFs, and using the same weight update rule as in [1].However, without experimental validation, it is difficult to definitively state whether this adaptation would further enhance [1]. When it comes to enhancing [2], which basically looks the problem in another way, more specifically the input; we firmly believe PSWFs can be incorporated into [2], as their proposal mechanism is based on the input. Detailed experiments are indeed needed to verify the claim. To further demonstrate the effectiveness of the proposed method, we compared the proposed approach with recently released FINER, INCODE, FR-INR on the entire Kodak image dataset. The following table summarizes the results. The suggested references, new methods, and additional results will be included in the revised version of the paper.

Table: Comparison of Methods Based on PSNR (dB)

First and foremost, the authors sincerely thank the reviewer for their thoughtful and insightful questions. Furthermore, we greatly appreciate your previous comments, which have been instrumental in refining our work and providing us with an excellent opportunity to further clarify and strengthen it.

W1:

The effective training times for the proposed method is as follows. As can be seen from the table, PIN's runtime is comparable with that of previous INRs.

Table: Convergence Time Across Methods

W2:

We thank the reviewer for their thoughtful question and careful review. We would like to clarify that the improved results were not obtained by fine-tuning our proposed method on specific datasets or scenes. Unlike existing baselines, our method uses the same configuration across all experiments, regardless of the data modality.

In the previous submission, we reported results for a 3D dataset, but it did not significantly contrast our results with other INRs. During the rebuttal for the previous submission, we noted that PIN performs well on all 3D datasets except the one used in that submission. Therefore, for this submission, we replaced the previously reported results with those from different 3D datasets to better showcase the performance gap of our method.

W3:

We are thankful for the reviewer for the question, and the suggestions. The authors acknowledge that we compared PIN with initial methods. However, to demonstrate the effectiveness of the proposed method, we compared the proposed approach with recently released FINER, INCODE, FR-INR on the entire Kodak image dataset. The following table summarizes the results. Further, the authors will make sure to cite the suggested and latest methods in the revised version of the manuscript.

Table: Comparison of Methods Based on PSNR (dB)

First and foremost, the authors would like to thank the reviewer for their thoughtful and insightful questions, which have provided us with an excellent opportunity to clarify and strengthen our work.

W1: We sincerely thank the reviewer for emphasizing the importance of larger-scale evaluations and for recognizing our use of the Kodak Lossless True Color Image Dataset. Evaluating INRs necessitates retraining the model for each image, making large-scale assessments computationally demanding. While many major baselines in INR research evaluate their methods on only two or three examples, we are, to the best of our knowledge, the first to conduct a comprehensive evaluation across the entire Kodak dataset. In addition, we extended our analysis to the DIV2K dataset as suggested, even though with some limitations. Due to time and resource constraints, we evaluated PIN and other state-of-the-art (SOTA) methods on a randomly selected subset of 30 images from DIV2K. The average PSNR values for this subset are presented in the table1 below, and as shown, PIN outperforms other methods on this subset as well. Given that these images were selected randomly, we believe this performance pattern is representative of PIN’s general superiority and would likely extend to the entire DIV2K dataset. Combining these results, we report the overall average PSNR metrics for 54 images (24 from Kodak and 30 from DIV2K) in table2. We deeply value the reviewer's suggestion regarding larger-scale evaluations and are actively considering this for future work. Specifically, leveraging meta-learning or other training efficiency mechanisms could make such evaluations more feasible. Thank you again for highlighting this aspect and for your constructive feedback. Further, we will incorporate these results to the revised manuscript.

Table 1: PSNR Variation across DIV2K dataset

Table 2: PSNR Variation Across DIV2K and Kodak Datasets

Q1:

We are thankful for the reviewer's question. We computed run-times for the convergence. The following table provides the run-times. As can be seen from the table, PIN's runtime is comparable with that of previous INRs.

Table: Convergence Time Across Methods

Q2:

We are thankful for the reviewer regarding the question. When it comes to INRs, they are based on the coordinates of the signal that is being provided to it. So for training the inpainting task, the coordinates corresponding to the inpainted regions are masked out, and during the testing time the entire coordinates of the image is provided. This will effectively asses the method's generalization abilities for unseen coordinates.