STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem

Dear Reviewer,

Thank you for your detailed and insightful review of our paper. We are very encouraged that you found our work to be well-written, our method to be clearly presented, and recognized its potential for meaningful impact. We sincerely appreciate the time you have taken to provide this valuable feedback. Your questions have highlighted several areas where we can provide further clarification and strengthen our contribution.

Below, we address the weaknesses and questions you raised.

On Comparison with Traditional Methods: Scalability, Cost, and Convergence (Weaknesses 2 & 3, Questions 1 & 2)

We thank you for raising these critical points regarding the comparison with traditional methods. This comparison is central to our work, and we have dedicated Section 4.4 ("Comparative Experiment with Traditional Algorithms") and Table 5 in our paper to a direct discussion and data presentation on this topic.

The Core Advantage: Overcoming the "Curse of Dimensionality"

It is essential to distinguish between the challenges of high-dimensional problems. For traditional methods like the Finite Difference Method (FDM), the primary bottleneck is not the algebraic eigenvalue solver itself (e.g., the Power Method or ARPACK), but rather the initial discretization of the continuous operator into a matrix. This process suffers from the "curse of dimensionality": to maintain accuracy in higher dimensions, the number of grid points must grow exponentially, leading to an intractable explosion in memory and computational cost.

Neural network-based methods and traditional algorithms excel in different domains. In low-dimensional problems, traditional methods are often faster and highly accurate. However, in high-dimensional settings, the cost of discretization makes them impractical, which is precisely where methods like STNet offer a significant advantage.

Analysis of Computational Cost and Memory Usage

Our experimental data clearly illustrates this trade-off:

• The Dilemma of Traditional Methods: As shown in Table 5 for the 5D Harmonic problem, for FDM to reduce the relative error of the first eigenvalue (λ1) from $3.20×10^{−1}$ to a modest $3.82×10^{−3}$, the memory consumption skyrockets from 0.0001 GB ($4^5$ grid) to 22.9 GB ($45^5$ grid). Even with this massive resource usage, the resulting accuracy is orders of magnitude lower than STNet's. The Efficiency of STNet: In contrast, STNet achieves a much higher accuracy of $4.62×10^{−5}$ while using only 1.14 GB of memory.

To further address your question, we provide a more detailed analysis of the convergence time and absolute error of the principal eigenvalue for a five-dimensional Harmonic problem. It is important to note that our traditional baseline in the paper uses scipy.eigs, which is based on the highly-optimized ARPACK library—a much more powerful solver than a basic power method.

This detailed data further validates our conclusion. For large-scale (high-dimensional) problems, STNet holds a significant advantage in both computational cost and memory usage. Its core strength lies in its mesh-free random sampling strategy, which leverages the expressive power of neural networks to approximate eigenfunctions directly. This approach completely bypasses the step of constructing and storing a massive matrix, thereby effectively circumventing the curse of dimensionality that plagues traditional grid-based methods.

The convergence process of all experimental solutions and their computational time and memory requirements will be added to the appendix of the subsequent official version.

On Applicability to Real-World Problems (Question 3)

This is an excellent question regarding the practical scope of our method. The iterative nature of STNet, which is inspired by the power method, requires the ability to perform an "operator action"—that is, to compute the result of the operator $\mathcal{L}$ acting on the current function approximation (the neural network's output).

The applicability of STNet to a given problem therefore hinges on this requirement:

• If the mathematical form of the operator is known, even if it's highly complex or the domain is irregular, the operator action can be precisely computed using automatic differentiation. In these cases, STNet is directly applicable and its mesh-free nature is a distinct advantage.
• If the operator is not analytically defined (e.g., it exists as a "black box" simulator) or the data is noisy, our framework could still apply as long as this operator action can be performed or simulated. However, if there is no way to evaluate the effect of the operator on a given function, then our method in its current form would not be applicable.

In summary, the ability to compute the operator action is the main prerequisite. Extending the framework to cases where the operator itself must be learned from noisy data (i.e., inverse problems) is a very important direction for future research.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

Thanks for providing detailed data. I have two questions after reading the reply.

1. Is a larger neural network needed for high high-dimensional system? And how does the number of parameters affect the convergence?
2. What data type was used in the neural network?

Dear Reviewer,

Thank you for your insightful follow-up questions. We greatly appreciate the opportunity to provide further clarification on these important aspects of our work.

On Network Size, System Dimensionality, and Convergence (Q1)

Yes, as you said, a larger neural network is generally required to effectively model higher-dimensional PDE systems, as the complexity of the target eigenfunction increases with the spatial dimension.

Specifically, as detailed in Appendix C.2 (Page 15) of our paper, we configured the STNet model with an increasing number of parameters for higher dimensions, consistent with the setup for the PMNN baseline. Each eigenfunction is approximated by its own MLP, with architectures as follows:

• 1D Problem: [1, 20, 20, 20, 20, 1]
• 2D Problem: [2, 20, 20, 20, 20, 1]
• 5D Problem: [5, 40, 40, 40, 40, 1]

Our research has shown that using an under-parameterized model (e.g., applying the 2D architecture to a 5D problem) fails to converge, as the network lacks the capacity to approximate the more complex high-dimensional function.

However, the relationship between parameter count and convergence accuracy is nuanced. Simply increasing the number of parameters for a fixed-dimensional problem does not guarantee better performance and can even be detrimental. To illustrate this, our ablation study on the 2D Harmonic problem (Appendix D, Table 8, Page 16) examined the effect of network width while keeping the depth and number of sample points constant. The absolute errors for the eigenvalues are reported below:

This data shows that for the 2D problem, increasing the network width (and thus, the parameter count) did not improve accuracy. In fact, our further experiments (Appendix D, Table 9) reveal that the number of sampling points is a more critical factor for convergence. When the number of samples is fixed, excessively increasing the model's parameters can lead to overfitting on the sampled points, which degrades the overall accuracy of the eigenfunction approximation.

In summary, we draw the following conclusions:

1. Higher-dimensional PDEs require larger networks to capture the increased functional complexity. An under-parameterized model will fail to converge.
2. For a fixed dimensionality and number of sample points, indefinitely increasing model parameters leads to overfitting and does not improve performance.
3. The optimal approach is to scale the model's parameters in correspondence with the problem's dimensionality and the number of sample points to ensure robust performance.

Crucially, because STNet inherits the architectural principles of the power method and PMNN [1], it is exceptionally parameter-efficient. For our 5D PDE experiments, STNet requires only ~5,000 parameters per eigenfunction, whereas leading baselines like NeuralEF [2] and NeuralSVD [3] require ~500,000. This significant difference further underscores the efficiency and superiority of our proposed architecture.

Thank you for raising this important point. We will incorporate this detailed analysis and these key findings into the appendix and the main experimental section of the final manuscript to enhance its clarity.

On Data Types Used in the Neural Network (Q2)

To ensure numerical precision and consistency across all our experiments, we used a uniform data type setting. Specifically:

• Neural Network Parameters: All model parameters in STNet were initialized and stored as torch.float64.
• Input Data: The input to the network is a list of coordinates for the sampling points within the domain, with each coordinate represented as a float64.
• Output Data: The network's output, which represents the value of the approximate eigenfunction at each sample point, is also a torch.float64 tensor.

All related code has been provided in the supplementary materials to ensure full transparency and reproducibility.

Thank you for this question. In the final version of our paper, we will add these specific implementation details, along with illustrative code snippets, to the appendix to further improve clarity.

References:

[1] Neural networks based on power method and inverse power method for solving linear eigenvalue problems, Computers & Mathematics with Applications, 2023.

[2] Neuralef: Deconstructing kernels by deep neural networks, ICML 2022.

[3] Operator svd with neural networks via nested low-rank approximation, ICML 2024.

We sincerely thank you once again for your detailed and constructive engagement with our work. We hope these detailed explanations fully address your remaining concerns. Should you have any further questions, please do not hesitate to let us know. We remain available for any further discussion. We are very grateful for your time and guidance, and we would be deeply appreciative if you would consider our clarifications in your final evaluation of our work.

Dear Reviewer btcB,

Thank you again for your thoughtful and constructive review.

Following up on our submitted rebuttal, we would be very grateful to hear your thoughts, especially as the discussion period is nearing its end. Your feedback is crucial for us to understand if our responses have adequately addressed your concerns and to help us further improve our paper.

We are happy to provide any further clarifications and look forward to discussing this with you.

Best regards,

Authors

Dear Reviewer,

Thank you for your valuable feedback and for recognizing the strengths of our work. We appreciate your positive assessment of our paper's clarity and the intuitive effectiveness of our proposed modules. We have carefully considered your suggestions and provide the following responses, which we hope will address your questions and further strengthen our submission.

On Convergence and Stability Analysis (Weakness 1)

We agree that a graphical illustration of convergence is essential for a complete analysis. To provide a preliminary view of the convergence behavior of STNet, we present the absolute error at different iteration stages for the Harmonic problem below. The empty cells in the table indicate that convergence for that particular eigenvalue was already achieved.

Absolute Error vs. Iterations for Harmonic Problem

Following your excellent suggestion, we will add comprehensive convergence plots (error vs. iterations) for all algorithms across all experiments in the final camera-ready version of the paper to provide a clearer visual analysis of the convergence dynamics.

On the Source of Performance Improvement and PMNN Comparison (Weakness 2)

This is an insightful question, and we thank you for the opportunity to clarify. You are correct in your observation. As stated on page 5, line 105, STNet is designed as an enhanced version of the Power Method Neural Network (PMNN). The PMNN framework itself is capable of achieving very high precision for a single eigenvalue.

Therefore, the "w/o F and D" case in our ablation study (Table 6) is functionally equivalent to the original PMNN. The high accuracy for the principal eigenvalue ($\lambda_1$) in this setting is indeed inherited from this strong baseline.

The primary limitation of PMNN, however, is its inability to compute multiple eigenvalues simultaneously. Our key contribution with STNet is the introduction of the deflation projection and filter transform, which extend the high-precision capabilities of the base method to solve for multiple eigenvalues concurrently and more accurately.

The effectiveness of our modules is clearly demonstrated in Table 6. In the "w/o D" setting, the network correctly finds the principal eigenpair $(v_1, \lambda_1)$ but fails to find the subsequent ones, with the errors for $\lambda_2$ and $\lambda_3$ being extremely large. This shows that the network repeatedly converges to the same dominant eigenfunction. The deflation projection module is what resolves this issue, enabling the network to discover multiple distinct eigenfunctions.

To make this clearer, we will explicitly state in the ablation study section of our final paper that the "w/o F and D" setting serves as a direct comparison to the PMNN baseline, and we will revise the text to better articulate how our contributions build upon it to overcome its limitations.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

Dear Reviewer,

Thank you for your thorough review and insightful comments on our paper. We sincerely appreciate the time and effort you have dedicated to providing this valuable feedback. We have carefully considered each of your points and offer the following clarifications and planned revisions. We hope these responses will fully address your concerns and lead you to reconsider your assessment of our work.

On the Clarity of the Abstract (Weakness 1)

We agree with your observation. Our intention was not to claim the invention of deflation projection and filter transforms. Our core contribution is to be the first to successfully integrate these powerful, classical spectral transformation techniques into a neural network framework for solving operator eigenvalue problems. As you noted, these methods are well-established in traditional numerical analysis, a point we detail in Sections 2.3 and 2.4 of our paper. To prevent any misunderstanding, we will revise the abstract in the final version to explicitly state that our work builds upon these classical numerical methods, clarifying our unique contribution.

On Theoretical Justification (Weakness 2)

This is an excellent point, and we appreciate the opportunity to clarify the theoretical underpinnings of our method. The key insight of STNet is that the spectral transformations are applied directly to the original operator $\mathcal{L}$, transforming it into an equivalent but easier-to-solve problem, $\mathcal{L''}$. This process, detailed in Sections 3.2.1 and 3.2.2, does not modify the neural network architecture or its learning mechanism.

Crucially, the transformation preserves the spectral properties in an exact, predictable way. It creates an equivalent problem, not an approximation. Therefore, if the neural network minimizes the loss to zero for the transformed problem $\mathcal{L''}$, it provides an exact solution to the original operator eigenvalue problem. The transformation only impacts the convergence path and final precision, not the validity of the solution. Our extensive experimental results, which demonstrate state-of-the-art accuracy, provide strong empirical validation for this theoretical consistency.

On the Implementation of the Differential Operator (Weakness 3)

Thank you for pointing out the need for more clarity here. The differential operator $\mathcal{L}$ is constructed and applied using automatic differentiation (AD). AD allows for the exact computation of derivatives of any order by decomposing functions into a sequence of elementary operations and applying the chain rule. For instance, to apply an operator like $\frac{d^2}{dx^2} + \frac{d}{dx}$, we use AD to compute the second derivative and the first derivative of the network's output with respect to its input and sum the results.

The specific computations for deflation and filtering (lines 7–8 in Algorithm 2 ) are detailed by the formulas in the main text, specifically Equation 9 (page 5, line 127) and Equation 11 (page 6, line 150).

We have also provided the complete, runnable code in the supplementary materials. For example, the operator application and deflation are implemented as follows:

Python

# Automatic differentiation to compute Laplaciandef compute_grad(self, u, x):
    u_x = autograd.grad(u.sum(), x, create_graph=True)[0]
    return u_x

u_xx = Nonefor i in range(self.d):
    # ... code to compute second partial derivatives and sum them ...
    u_xixi = self.compute_grad(self.compute_grad(u, x[i]), x[i])
    u_xx = u_xx + u_xixi if u_xx is not None else u_xixi

# Application of deflation projection operatordef apply_deflation(u_xx, u, max_vecs, max_vals):
    Lu = -u_xx
    # Sequentially subtract projections onto solved eigenfunctionsfor i in range(len(max_vecs)):
        projection = max_vals[i] * torch.matmul(max_vecs[i], torch.matmul(max_vecs[i].transpose(0, 1), u))
        Lu = Lu - projection
    return Lu

We will incorporate these explanations, along with more detailed pseudocode and illustrative code snippets, into the main paper to make our implementation crystal clear. We also plan to release our final code as a user-friendly library to facilitate broader use.

On Details of the Experimental Setup (Weakness 4)

We apologize for any lack of clarity. The loss function is explicitly defined in our paper. On page 4, Algorithm 2, line 10, the loss for each network is given as: $Loss_{i}^{k} = \frac{1}{N}\sum_{j=1}^{N} [\tilde{v}_i^{k-1}(x_j)-\tilde{u}_i^{k}(x_j)]^{2}$.

This loss formulation remains consistent across all experiments. The only part that changes is the definition of the operator $\mathcal{L}$ within the computation of $\tilde{u}\_{i}^{k}$. For full transparency, we will add the specific code snippets for implementing the operator $\mathcal{L}$ for each problem (Harmonic, Schrödinger, Fokker-Planck) to the appendix in the final version. For instance, the implementation of Harmonic operator is:

Python

# Code snippet for Harmonic operator L
x = self.x
u = self.forward(x)
u_xx = Nonefor i in range(self.d):
    # ... (AD code as shown in previous point)
Lu = -u_xx # Lu represents the action of the operator L on u

On the Organization of Main Content and Appendix (Weakness 5)

This is a very helpful suggestion. We agree that the background on deflation projection and filter transform, as well as a more detailed comparison of the baseline methods, are crucial for understanding our paper's context and contribution. In the camera-ready version, we will move these essential details from the appendix into the main body of the paper to improve readability and flow.

On the Method's Applicability to Non-linear Operators (Question 2)

The paper focuses on linear operators primarily because this is the most fundamental and intensively studied area for neural network-based eigenvalue solvers. However, this focus represents an implementation choice rather than a hard structural limitation. In traditional numerical analysis, many non-linear and quadratic eigenvalue problems are tackled by reformulating them into a sequence of linear problems that are solved iteratively.

Following this established paradigm, STNet can, in principle, be extended to solve these more complex classes of problems. We see the current work as a foundational step, and we plan to develop optimized versions of STNet for non-linear, generalized, and other specific eigenvalue problems in our future research.

On Comparisons with Advanced Traditional Methods (Question 3 & 4)

This is an important question. As we discuss on pages 8-9 (lines 248-254), the primary bottleneck for traditional methods like FDM in high dimensions is the discretization error, not the algebraic eigenvalue solver's error. Techniques like deflation or filtering can improve the accuracy of the matrix eigenvalue solve, but they cannot overcome the fundamental error introduced by discretizing a high-dimensional space onto a grid. Reducing this error requires increasing the grid density, which leads to an exponential increase in memory and computational cost, a problem known as the "curse of dimensionality".

To further validate this point, we have run a new comparison for the 5D Harmonic problem against FDM with Richardson extrapolation, a technique that improves accuracy. The results, shown below, confirm that STNet still achieves significantly higher accuracy with comparable memory usage.

This experiment reinforces that neural network-based methods, which rely on random sampling rather than fixed grids, are inherently better suited for high-dimensional problems. We will add this experiment and a more detailed analysis to the appendix in our final version.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

We sincerely apologize for the typographical error in our response to Questions 3 & 4.

The values for "Grid/Points" in the table should be "$25^5$, $25^5$, $45^5$, $45^5$, $9^5$".

This was an oversight on our part during the format conversion process. We deeply regret any confusion this may have caused.

We greatly appreciate your constructive feedback and your willingness to raise your score! We are pleased to hear that our responses and the new FDM comparison have helped resolve your primary concerns.

We also acknowledge your valuable suggestions for improvement. In the final version, we will focus on enhancing the theoretical justification and clarifying key implementation details to make the paper more self-contained, as you recommended. We will, of course, also incorporate the newly added results. Thank you again for your guidance.

Dear Reviewer,

Thank you for your thoughtful and constructive feedback on our paper. We are grateful for the time you have taken to review our work and for your valuable suggestions, which have helped us to identify areas for improvement. We have carefully considered all your comments and provide the following clarifications, which we hope will address your concerns. We believe these explanations will further strengthen our paper, and we hope you will consider raising your score.

On Network Architecture and Ablation Studies (Weaknesses 1)

As noted on page 5, line 108 of our paper, the neural network modules employed in our work are all Multi-Layer Perceptrons (MLPs). The core contribution of STNet lies not in a novel network architecture, but in the use of Deflation Projection and Filter Transform to modify the loss function and the optimization process. For this reason, and to ensure a fair comparison with baseline methods, we adopted the standard MLP architecture used by NeuralSVD and PMNN.

To analyze the impact of the architecture, we conducted ablation studies on the depth and width of the MLP, with the results presented in the appendix on page 16 (Tables 7 and 8). We agree that designing a network architecture more specifically suited to this task is a promising direction for future research.

On the Dependency on Initial Eigenvalue Estimates (Weaknesses 2)

STNet does not critically depend on a precise initial estimation of the eigenvalues. A key feature of our method is that the Filter Transform module dynamically adjusts its filter function based on the approximate eigenvalues computed by the network at each iteration. This adaptive process progressively refines the search and is the primary reason STNet achieves higher accuracy than other algorithms. In all our experiments, we used broad, non-specific initial ranges such as [-10, 10] or [-100, 100], demonstrating the method's robustness.

On Computational Complexity and Runtimes (Weaknesses 3 & 4)

We appreciate your concern regarding computational complexity. As detailed on page 6, lines 144-151, our method circumvents the high cost of direct matrix inversion. We simulate the inverse power iteration process by reformulating the loss function, thus avoiding explicit and computationally demanding matrix inversions.

While it is true that neural network-based methods can have a larger computational overhead than traditional methods in low-dimensional problems (as mentioned on page 8, line 248), they offer a distinct precision advantage in high-dimensional settings. We provide a runtime comparison for the 5D Harmonic problem below:

This comparison highlights two points:

1. STNet achieves a lower error more rapidly and converges to a significantly more accurate solution than NeuralEF and NeuralSVD.
2. The final convergence time for STNet is longer. This is an inherent trade-off of the Filter Transform, which iteratively refines the problem to unlock further optimization, thus requiring more iterations to reach its high-precision potential.

We will include detailed runtime records for all experiments in the final version of the paper.

On the Rate of Convergence (Weaknesses 5)

We thank you for this suggestion. A full theoretical analysis of the convergence rate is indeed challenging. To provide empirical evidence, we present the convergence process for the Harmonic problem below. The empty cells indicate that convergence for that eigenvalue has already been achieved.

Absolute Error vs. Iterations for Harmonic Problem

Given the space constraints of the rebuttal, we will add comprehensive convergence curves (error vs. iterations and time) for all algorithms in the final manuscript.

On Grid Resolution Analysis (Weaknesses 6)

In our main experiments, we fixed the number of sampling points to ensure a fair and direct comparison between methods, with the specific parameters detailed in Appendix C.2 on page 15.

To analyze the effect of this hyperparameter on STNet, we performed a dedicated analysis by varying the number of sampling points, as presented on page 16, line 511 (Table 9). The results in Table 9 show that as long as the number of sampling points is not set too low, STNet maintains its strong performance, demonstrating its robustness to this parameter.

On Updating the Related Work Section (Weaknesses 7)

Thank you for this valuable suggestion. We have conducted a further literature search, including a review of recent citations for key papers like PMNN and NeuralEF. Our search identified several recent papers that focus on optimizations for specific eigenvalue problem scenarios. However, we did not find new works proposing general-purpose methods for computing differential operator eigenvalues that would constitute direct baselines. We will gladly incorporate these newly found, relevant papers into the related work section in the final version to provide a more current context.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

We greatly appreciate your constructive feedback and your willingness to raise your scores! We are pleased to hear that our responses have resolved most of your concerns. As suggested, we will incorporate the newly added results into the final version of the paper.