Accelerating Eigenvalue Dataset Generation via Chebyshev Subspace Filter

Dear Reviewer,

Thank you for your thoughtful and constructive review. Your feedback is immensely helpful for strengthening our paper, and we offer the following responses to address your concerns.

On Theoretical Insights, Applicability, and Failure Cases (W1, W3, Q1, Q2)

Thank you for these insightful points. The performance of SCSF hinges on the continuous relationship between operator parameters and the resulting matrix properties, a common characteristic in datasets for scientific ML where parameters are generated smoothly (e.g., via Gaussian Random Fields)[1-3]. In this setting, our sorting algorithm effectively groups similar problems, maximizing acceleration. Conversely, if the problems are independent or discontinuous, the sorting provides little benefit, and SCSF's performance gracefully degrades to that of a warm-started Chebyshev filter.

To provide concrete insight, we conducted two supplementary experiments:

Experiment 1: Quantifying the Impact of Similarity. We simulated datasets with controlled similarity by perturbing the parameters of a Helmholtz operator. The "perturbation size" indicates the degree of dissimilarity between consecutive problems.

Note: This experiment was run on the Helmholtz dataset (dim=6400, L=200). Times are in seconds. The latter problem is obtained by perturbing the parameters of the former. The perturbation is obtained by multiplying the perturbation size by the Gaussian random field. The perturbation size can be considered a measure of the internal similarity of the dataset. The "standard method" is the same as the experimental setup in the paper. "Independent Init." can be considered a special case where the operator parameters and the resulting matrix are completely independent and discontinuous.

This shows: 1) Performance is strongly correlated with problem similarity. 2) Our sorting algorithm is highly effective at enhancing this correlation (SCSF vs. SCSF w/o sort). 3) In the worst case of completely independent problems, SCSF degrades to ChFSI, as expected.

Experiment 2: Simulating a Discontinuous Failure Case. We mixed two different datasets (Helmholtz and Poisson, 1:1 ratio) to simulate a discontinuous problem sequence.

This shows that SCSF's performance gain is reduced when correlation is disrupted, and the benefit of sorting also decreases, aligning with our expectations.

Regarding adaptive sorting strategies: This is an excellent suggestion. While not explored in the current work, we envision future extensions where the solving order could be dynamically adjusted based on runtime information. This would be particularly useful for online settings where data is acquired sequentially (e.g., from real-time sampling or iterative data generation guided by model performance). This is a promising direction for our future research.

Action: We will incorporate these experiments and a more detailed discussion on applicability and failure modes into Sections 3 and 5 of the revised manuscript. We will also expand the limitations section to explicitly state that SCSF is designed for datasets with underlying continuity.

On the Overhead and Cost-Benefit of Sorting (W2 & Q4)

Thank you for raising this important point about the computational trade-offs.

• Sorting Overhead Analysis: The overhead of our sorting algorithm is minimal in practice. As shown in Table 4, sorting a large dataset of 10^4 samples takes approximately 151 seconds. In contrast, solving a single eigenvalue problem on the Helmholtz dataset (400 eigenvalues) takes an average of 253 seconds with Eigsh. For a dataset of this size, the total sorting overhead constitutes less than 0.1% of the total generation time, making its cost negligible.
• Benefit and Break-Even Point: The benefit of sorting is a significant reduction in the number of solver iterations and total floating-point operations (Flops). Table 3 shows that sorting reduces iterations by 5-50% and total Flops by 7-43%. Given that the sorting cost is a tiny fraction of the total computation, a "break-even point" is not a practical concern for typical dataset sizes. The more relevant goal is to find the fastest sorting algorithm that still effectively captures the similarity structure. Our Truncated FFT Sort achieves this, providing nearly identical solver performance as the far more expensive greedy sort (Table 5), demonstrating its cost-effectiveness.

Action: We will add a more detailed analysis, potentially with a plot illustrating the relationship between sorting overhead and total time saved, to the appendix of the revised paper.

On Experimental Depth and Hyperparameter Analysis (W3)

We appreciate your feedback on making the hyperparameter analysis more accessible. We conducted a detailed analysis of key hyperparameters (filter degree m, subspace dimension, and truncation threshold p_0) in Appendix D.4. Our main finding is that SCSF is robust to these hyperparameters within reasonable ranges. For instance, varying the filter degree m between 16 and 40 has a minimal effect on computation time , and setting the truncation threshold p_0 to a small value (e.g., 20) achieves solver performance nearly identical to that of the costly full greedy sort.

Action: To improve the main paper's clarity, we will move a concise summary of these key findings on hyperparameter robustness from the appendix into the main experimental section, while keeping the detailed tables and analysis in the appendix for interested readers.

On Motivation for Method Choice and Significance (W4 the clarity front)

Thank you for these two key points.

• Why Chebyshev Filtering? Our choice was guided by the empirical results presented in our ablation study in Section 4.3. As shown in Table 2, when we modified other standard solvers (Eigsh, LOBPCG, KS, JD) to use a "warm start" from the previous solution, only LOBPCG saw a notable speedup, while SCSF (which uses Chebyshev filtering) was significantly faster than all of them. The mechanism behind this is that the Chebyshev filter, via a polynomial transformation, is exceptionally effective at amplifying the components of the target invariant subspace while suppressing others. This makes it uniquely suited to capitalize on a high-quality initial subspace provided by a similar, previously solved problem.
  • Action: We will add a more explicit motivation for this choice in Section 3.2.
• Why Focus on Dataset Generation? Our core contribution is not a new general-purpose eigensolver, but rather an efficient and lossless optimization of traditional solvers for the specific, emerging problem of dataset generation in AI. This addresses a widely recognized bottleneck in "AI for Science." As a leading survey notes [4] P169, "A primary limitation of learned solvers is the requirement of an adequate number of training data generated by costly numerical solvers". An efficient generation tool lowers the barrier to entry and accelerates the research cycle for the entire community. Furthermore, the core idea—transforming a batch of related problems into a "sequential problem" to be solved with warm starts—has broader potential in fields like optimization, signal processing, and stability analysis, which we plan to explore in future work.

On Extending SCSF to Other Eigenvalue Problems (Q3)

This is an excellent, forward-looking question. Extending SCSF to non-Hermitian, generalized, or nonlinear eigenvalue problems is an important direction for future work. Our current thinking involves:

1. Adapting the Solver: We would need to employ or develop Chebyshev-like iterative methods suitable for these other problem types. Versions for generalized eigenvalue problems already exist. For nonlinear problems, a common approach is to linearize them into a sequence of linear eigenvalue problems, a process where the SCSF paradigm could be applied to accelerate the solution of the sequence.
2. Designing New Sorting Metrics: Our current sorting relies on the distance between parameter matrices. This may not be optimal for all problem types. New, specially designed metrics would likely be needed to effectively capture problem similarity in these different contexts.

Action: Based on your suggestion, we will expand this discussion in the revised version to also include the potential for extension to non-Hermitian and generalized eigenvalue problems, providing a more comprehensive view of our work's current scope and future directions.

References:

[1] Fourier neural operator for parametric partial differential equations, ICLR 2019.

[2] Neural krylov iteration for accelerating linear system solving, NIPS 2024.

[3] Accelerating data generation for neural operators via krylov subspace recycling, ICLR 2024.

[4] Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems, 2023.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

We greatly appreciate your constructive feedback! We are pleased to hear that our responses have resolved most of your concerns. As suggested, we will incorporate the newly added results into the final version of the paper.

Dear Reviewer,

We sincerely thank you for your positive feedback on our paper's clarity and the intuitive nature of our approach. We are grateful for your thoughtful comments, which give us the opportunity to further clarify the significance and positioning of our work. We hope our response below will address your concerns and persuade you of our paper's value to the NeurIPS community.

On the Fit of Our Work for NeurIPS (W1)

We greatly appreciate your perspective on the most suitable venue for our work. While our SCSF method is indeed rooted in numerical analysis techniques, we wish to emphasize that its motivation, application, and intended audience are firmly centered within the machine learning **community, particularly the rapidly growing field of "**AI for Science."

1. Motivation Stemming from the AI Community: Our primary motivation, as you correctly noted, is to accelerate the generation of labeled datasets required to train data-driven models like [1] ICLR 2019, [2] NIPS 2024 spotlight. NeurIPS has become a premier platform for showcasing such cutting-edge AI for Science research. Our work directly addresses a critical bottleneck faced by this community, aiming to provide a foundational tool that enables more efficient research.
2. Serving the Needs of the AI Community: The challenge we address is widely recognized within the field.
   1. As noted by Prof. Max Welling et al. in their ICML 2022 paper [3], the AI community faces a "proverbial ‘chicken-and-egg problem’": training neural solvers requires abundant data that must first be generated by classical solvers.
   2. A comprehensive survey on AI for Science [4] (P169) highlights this directly: "A primary limitation of learned solvers is the requirement of an adequate number of training data generated by costly numerical solvers."
   3. Our research aims to lower this barrier to entry. Currently, generating a large-scale eigenvalue dataset can take days or even weeks. By dramatically accelerating this process, our method enables researchers, especially those with limited access to large-scale computing resources, to participate more actively in this field, thereby fostering innovation across the entire AI for Science ecosystem.
3. Bridging Two Fields: We believe that innovatively applying established numerical analysis techniques (like Chebyshev filtering) to solve practical problems in machine learning is precisely the kind of interdisciplinary research that NeurIPS champions. Our work not only provides a tool for the AI community but also showcases a novel application scenario for numerical analysis techniques in the AI era, serving as a valuable bridge between the two domains.
4. Precedent at Top AI Venues: There is a strong precedent for work on accelerating dataset generation at top AI conferences. Recent papers at ICLR 2024 spotlight [5] and ICML 2024 [6] have focused on accelerating the solution of linear systems for this purpose. Our paper builds on this emerging topic, and to our knowledge, is the first to specifically address the acceleration of eigenvalue dataset generation, a distinct and equally important challenge.

For these reasons, we are confident that NeurIPS is the ideal platform to present this work, as it directly confronts a pressing issue faced by the conference's core audience.

On the Significance of the 3.5x Speedup (W2)

Thank you for this insightful question regarding the importance of our method's performance gains. We would like to elaborate on the significance of our work from three perspectives.

1. The 3.5x Speedup is a Conservative Estimate: The "3.5x" figure mentioned in the abstract is a conservative summary that holds true across all tested scenarios. In many specific cases, the performance gains were far more substantial. For instance, on the Helmholtz dataset, SCSF achieved speedups of up to 8x, 20x, 6x, and even 95x compared to various state-of-the-art solvers. Reducing computation time from weeks to days represents a transformative improvement in research efficiency and iteration speed.
2. Dataset Generation is Not a "One-Off" Task: In the practice of AI for Science, data generation is a recurring and essential part of the research lifecycle.
   1. Diverse Scientific Problems: Different physical phenomena require unique, customized datasets. The variety of possible differential operators is immense, meaning there will always be a need to generate new data for new problems. This is a primary driver for research in data generation and augmentation.
   2. Iterative Research and Exploration: Scientific inquiry is an iterative process. Researchers frequently adjust parameter ranges, discretization precision, or operator forms to explore physical laws under different conditions. Each adjustment typically requires generating an entirely new dataset.
   3. Model and Hyperparameter Tuning: Even for a single problem, the process of finding an optimal model architecture and its hyperparameters often involves evaluation on datasets of varying scales or distributions.
   4. Therefore, an efficient data generation pipeline is a critical enabler for the entire iterative AI for Science research paradigm.
3. Broader Application Horizons: You have keenly pointed out the potential for sequential eigenvalue solving in other domains. Beyond dataset generation, this task is crucial in numerous scientific and engineering fields where sequences of similar matrices arise. Potential applications include:
   1. Data analysis in machine learning and optimization, where sequences of closely related matrices are often analyzed.
   2. Signal processing in communications and stability analysis in power systems, which frequently involve solving for eigenvalues of highly similar matrices.

In our future work, we plan to adapt the core ideas of SCSF to these domains, transforming large sets of eigenvalue problems into sequential tasks to be solved efficiently.

In summary, our work provides a significant performance improvement for a critical and recurring task in AI for Science, and its core methodology holds promise for empowering a much broader range of scientific and engineering applications.

References:

[1] Fourier neural operator for parametric partial differential equations, ICLR 2019.

[2] Neural krylov iteration for accelerating linear system solving, NIPS 2024.

[3] Lie Point Symmetry Data Augmentation for Neural PDE Solvers, ICML 2022.

[4] Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems, 2023.

[5] Accelerating data generation for neural operators via krylov subspace recycling, ICLR 2024.

[6] Accelerating pde data generation via differential operator action in solution space, ICML 2024.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

Dear Reviewer QhCL,

Thank you very much for taking the time to read our rebuttal and for promptly acknowledging it. We truly appreciate your engagement with our work during the review process.

If you have any further thoughts, questions, or suggestions---either regarding our current response or the broader direction of the work---we would be genuinely grateful to hear them. We highly value your perspective and would welcome any opportunity for continued discussion or clarification.

Best,

Authors

Dear Reviewer QhCL,

Thank you again for your thoughtful and constructive review.

Following up on our submitted rebuttal, we would be very grateful to hear your thoughts, especially as the discussion period is nearing its end. Your feedback is crucial for us to understand if our responses have adequately addressed your concerns and to help us further improve our paper.

We are happy to provide any further clarifications and look forward to discussing this with you.

Best regards,

Authors

Dear Reviewer,

Thank you for your constructive feedback and for recognizing our work's relevance and empirical performance. Your insightful questions have helped us clarify our contributions. We hope our responses below adequately address your concerns and will encourage you to reconsider your score.

On the Novelty of Our Work (W1 & Q1)

Our core innovation is not merely combining existing techniques, but the first systematic formulation and solution for accelerating operator eigenvalue dataset generation.

Our novelty is threefold:

1. Problem Formulation and Framework: Traditional methods treat each eigenvalue problem in a dataset as an isolated task, leading to significant computational redundancy. We are the first to reframe "eigenvalue dataset generation" as a "sequential eigenvalue solving" problem. Based on this new perspective, we propose a novel "Sort-Solve" framework designed to maximally exploit the intrinsic similarities among operators.
2. Systematic Methodological Integration: As shown in our experiments (Sec 4.4, Page 9, Line 284), ChFSI performs poorly on unsorted, disordered problems, sometimes even slower than standard solvers. Likewise, sorting is of little value without an efficient solver that can leverage the created sequence. Our key contribution lies in the systematic integration of an efficient "Truncated FFT Sort" with a "Chebyshev filter" optimized for sequential problems. The Truncated FFT Sort (Algorithm 2) enhances the correlation within the problem sequence at a very low computational cost (far cheaper than greedy sorting), creating the ideal conditions for the Chebyshev filter to thrive. This creates a synergistic effect where the whole is greater than the sum of its parts.
3. Significant Empirical Value: As shown in our ablation study (Sec 4.3, Page 8, Line 246), we experimented with combining sorting with various traditional algorithms and found that the combination in SCSF delivered the most stable and significant performance gains. SCSF achieves state-of-the-art performance across four distinct datasets, with speedups ranging from 3.5x to 95x. This demonstrates the effectiveness of our proposed framework and provides a practical, valuable tool for the AI for Science community to overcome the data generation bottleneck.

Therefore, our novelty lies not in a single component, but in the new problem formulation, the systematic design of our integrated method, and the exceptional performance it achieves.

To directly answer your question, we consider the "Truncated FFT Sort" (Algorithm 2) to be the most novel component of our methodology. Its ingenuity is rooted in a key insight: the spectral properties of operators are primarily influenced by the low-frequency components of their parameters. By performing an FFT on the parameter matrices and sorting based only on these retained low-frequency components, we achieve:

• High Efficiency: The computational complexity of FFT is O(p^2logp), far more efficient than a greedy algorithm that directly computes distances on high-dimensional parameter matrices. As shown in Table 4, for a dataset of size 104, our sorting algorithm is nearly 4 times faster than greedy sorting.
• Excellent Effectiveness: As shown in Table 5, the final solution efficiency achieved using our low-cost sorting method is nearly identical to that achieved using the computationally expensive greedy sort.

This "low-cost, high-yield" sorting strategy is the key enabler for our SCSF framework and represents a significant and novel methodological contribution.

On Writing and Structure (W2)

Thank you for the valuable suggestion. We agree Section 2.2 was abrupt. In the revision, we will add a transitional paragraph to motivate the need for Chebyshev polynomial preliminaries and move most of the technical derivations to the appendix, keeping only the core definitions in the main text to improve flow.

On Optimality Properties and Speed Bounds (Q2)

This is a very insightful question. From a theoretical standpoint, our method does not currently possess a formal proof of "optimality," and its performance upper bound is difficult to define with a single formula, as it is highly dependent on the intrinsic properties of the dataset.

• On Optimality: The performance of SCSF is strongly correlated with the "similarity" among operators in the dataset. In an ideal scenario where two consecutive matrices, A(i−1) and A(i), are nearly identical, using the eigenvectors of A(i−1) as the initial subspace for A(i) could lead to convergence in just one or two Chebyshev filter iterations. Our sorting algorithm is a heuristic approach designed to approximate an optimal sequence that maximizes this similarity.
• On Speed Bounds: The upper limit of the speedup depends on two factors: (1) Post-sorting similarity: Higher similarity leads to fewer iterations and greater acceleration potential. (2) Filter operation overhead: As shown in Table 11, the Chebyshev filter step itself consumes over 70% of SCSF's total computation time. Therefore, even if the number of iterations were reduced to a minimum (e.g., 1), the total speedup is bounded by the fixed cost of this single iteration.

To better illustrate the theoretical performance ceiling, we conducted a simulation. We generated a sequence of Helmholtz operator problems where each subsequent problem is a slight perturbation of the previous one. The magnitude of this perturbation reflects the dataset's internal similarity. The experiment was run on the Helmholtz dataset (dim=6400, L=200). Average solution times (in seconds) are reported.

Standard Generation refers to the setup in our paper. Independent Problems simulates a case with no exploitable similarity, where each solver is initialized independently.

This experiment shows:

1. The theoretical performance limit is approximately 2 seconds, corresponding to 1-2 subspace iterations.
2. Performance is heavily influenced by dataset correlation. The more related the problems are, the greater the advantage of SCSF and ChFSI.
3. The sorting algorithm in SCSF is highly effective at enhancing problem correlation, as the performance of SCSF is consistently better than SCSF w/o sort until the problems are already highly correlated.
4. If the problems are completely independent, SCSF's performance degrades to that of ChFSI, as expected.

Action: In the final version, we will incorporate this experiment and analysis to provide a clearer picture of our method's performance bounds.

On the Choice of Experimental Datasets (Q3)

Our choice of four datasets was deliberate, aiming for comprehensive evaluation.

Representativeness:

The datasets cover key differential operators in science and engineering (Poisson, Elliptic, Helmholtz, Vibration). These operators vary in their mathematical properties (e.g., order) and physical origins, providing a robust testbed for our method's universality. Furthermore, these operators and their corresponding PDE problems are commonly used as benchmarks in mainstream neural operator papers [1, 2, 3], ensuring the relevance of our work to the community.

Challenging Problems:

Our method assumes a continuous relationship between operator parameters and matrix properties, which holds for most data generation scenarios. If this assumption fails (e.g., the problems are discontinuous), the sorting module becomes ineffective, and SCSF performance degrades to the "w/o sort" version. Even so, as shown in Table 3, the "w/o sort" SCSF is still 1.2 to 1.5 times faster than a randomly initialized ChFSI, demonstrating that simply using the previous solution as an initial guess provides a benefit.

To further illustrate this, we simulated an extreme case by mixing the Helmholtz and Poisson datasets (1:1 ratio, dim=6400, L=200), creating a discontinuous sequence.

The results confirm our expectations:

1. SCSF's performance gain is reduced in this discontinuous scenario because the internal correlation is disrupted.
2. The effectiveness of sorting is diminished, as the parameters no longer reliably predict matrix similarity across dataset types.

In summary, SCSF is most effective in the common scenario where dataset generation involves continuous parameter variation.

Action: We will add this experiment and analysis to the final version. We will also explicitly state this assumption in the limitations section and recommend that users assess the continuity of their data generation process before applying SCSF.

References:

[1] Fourier neural operator for parametric partial differential equations, ICLR 2019.

[2] Neural krylov iteration for accelerating linear system solving, NIPS 2024.

[3] Accelerating data generation for neural operators via krylov subspace recycling, ICLR 2024.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

We greatly appreciate your constructive feedback and your willingness to raise your scores! We are pleased to hear that our responses have resolved most of your concerns. As suggested, we will incorporate the newly added results into the final version of the paper.

Dear Reviewer,

Thank you for your detailed and insightful feedback. We have carefully considered your comments and provide our responses below, which we hope will clarify our contributions and address your concerns.

On the Significance of Accelerating Dataset Generation (W1)

Thank you for the valuable references to unsupervised methods [1, 2, 3]. While these represent an important research direction, our work focuses on the dominant supervised paradigm in scientific computing. Methods like FNO [4] and NeurKItt [5] require large, high-precision labeled datasets, and the significant computational cost of generating this data is a primary bottleneck. Our goal is to provide a crucial acceleration tool for these widely used methods. With speedups of 3.5x and often significantly higher, SCSF can reduce data generation cycles from weeks to days, greatly improving research efficiency.

The two approaches have different scopes. Unsupervised methods are often slow and less accurate for the common low-dimensional problems found in engineering. Our supplementary experiment on a 2D Helmholtz problem illustrates this trade-off (solving for the first 100 eigenvalues, matrix dimension 6400).

Note: As SPIN is an earlier, slower algorithm, we compared against its improved versions, NeuralEF and NeuralSVD, using their official code. For a single problem, SCSF's initial vector is randomly initialized.

This experiment demonstrates that each approach has its place. Unsupervised methods avoid the need for data but are slow and less accurate for these problems. Supervised methods are the fastest at inference but require significant training and data generation time. This validates the importance of developing supervised methods and, consequently, the significance of accelerating their dataset generation.

Action: In the final version, we will add a discussion of unsupervised methods to our "Related Work" section to better frame our contribution.

On Methodology: "Ground Truth" and Scalability (W2)

Thank you for your sharp insights. We wish to clarify the reliability and scalability of our method.

• On the Reliability of Datasets as "Ground Truth":
  • In the machine learning context, "Ground Truth" refers not to an unobtainable analytical solution, but to a reference solution with precision far exceeding the predictive capability of the neural model being trained. Our data generation pipeline involves two steps: 1) operator discretization and 2) matrix eigenvalue solving. Our paper's core contribution is accelerating the second step, which is the computational bottleneck, often consuming over 95% of the total time.
  • In all our experiments, the relative residual for the matrix solve (for both baselines and SCSF) is set to a high precision of 1e−8 to 1e−12. Our tests confirm that the discretization error for all experiments is well below 1e−6. This level of accuracy is orders of magnitude higher than the typical error range of neural network like FNO [4] or NeurKItt [5], which is usually between 10−3 and 10−5. Therefore, the datasets generated with our method are a highly reliable "Ground Truth" for training such neural networks.
• On Discretization Sensitivity and Scalability:
  • We fully agree that the final accuracy and scale of matrix-based methods are tied to the initial discretization. However, our focus is not on the discretization technique itself, but on efficiently solving the resulting matrix problems.
  • Discretization-Agnostic: SCSF method is agnostic to the choice of discretization. Whether a spectral method is used, SCSF can accelerate the process as long as it results in a sequence of similar matrix eigenvalue problems.
  • Standard and Relevant Discretization**:** The discretization methods and code used in our experiments are adopted directly from the benchmark dataset generation procedures of leading works like [4], [5], [6]. They use the exact same procedures to generate their "Ground Truth" training and test sets, ensuring our work is directly relevant and compatible with current best practices.
  • Performance on High-Dimensional Problems: Your point about high-dimensional PDEs is crucial. These problems result in matrices with very large dimensions, making them challenging for traditional solvers. This is precisely where SCSF provides the most value. Our results in Figure 3 and Table 10 clearly show that SCSF's performance advantage grows as the matrix dimension increases. This demonstrates that our method is even more critical for tackling the large-scale, complex problems that are most relevant to practical applications, helping to push past existing computational barriers.

To further solidify this point, we trained NeurKItt [5] using generalized Poisson datasets generated by different solvers and at different matrix dimensions. The solver precision was set to 1e-12.

Note: To ensure a fair comparison, all data was interpolated to a 100x100 grid before training.

The experiment shows that for matrix dimensions of 2500 and above, the choice of solver and the exact dimension have a negligible impact on the final trained model's performance. This is because the data precision is already far beyond what the neural network requires, confirming its status as a reliable "Ground Truth."

Action: We will add the results of these supplementary experiments and a detailed analysis of discretization versus solver error to the final version of our paper.

On Performance Gain and Parallelization (W3)

• Significance of the 3.5x Speedup: The "3.5x" figure mentioned in the abstract is a conservative summary. As shown in Table 1, our performance gains are often much more substantial. For instance, on the Helmholtz dataset with 600 eigenvalues, SCSF achieves speedups of approximately 8x, 20x, 6x, and 95x over Eigsh, LOBPCG, KS, and JD, respectively. This level of efficiency gain, which can shorten computation times from weeks to days, is of great practical importance for accelerating the scientific research cycle.
• Relationship with Parallelization: SCSF and parallelization are orthogonal and synergistic. Our method accelerates the processing of a problem sequence on a single core. In practice, one can partition the N problems into M chunks and run SCSF in parallel on M cores, multiplying the speedup. All our experiments were conducted under fair, identical parallel conditions.

Action: We will add a clear explanation to the final manuscript clarifying that our method is complementary to parallelization and that the reported speedups are achieved under fair, parallelized conditions.

On Handling Boundary Conditions (Q1)

We use Dirichlet boundary conditions, incorporated during the matrix assembly stage, following standard procedures from benchmarks like [4-6].

Action: We will clarify this in the revision and release our data generation code.

On the Effect of FDM Parameters on Error (Q2)

Our focus is on accelerating the matrix solve, not optimizing discretization parameters like grid spacing. To ensure fairness and relevance, we adopted discretization code from established benchmarks and tested our method across a wide range of matrix dimensions (2500 to 10000) to demonstrate SCSF's robustness and applicability in realistic dataset generation scenarios.

On Other Comments:

Thank you for your constructive suggestions. We will revise Eq. (2) with a more illustrative 3x3 example, correct the typo on L86, move the bulk of Sec 2.2 to the appendix to improve flow, and expand our Limitations section to include a discussion on the intrinsic errors from discretization, as you recommended.

References:

[1] Spectral Inference Networks: Unifying Deep and Spectral Learning, ICLR 2019.

[2] Neuralef: Deconstructing kernels by deep neural networks, NIPS 2022.

[3] Operator SVD with neural networks via nested low-rank approximation, ICML 2024.

[4] Fourier neural operator for parametric partial differential equations, ICLR 2019.

[5] Neural krylov iteration for accelerating linear system solving, NIPS 2024.

[6] Pdebench: An extensive benchmark for scientific machine learning, NIPS 2022.

Thanks again

We sincerely thank you again for your constructive and detailed feedback, which has helped us identify clear pathways to improve our paper. Should you have any further questions or require additional discussion, please don't hesitate to reach out. If we have adequately addressed your concerns, we would be grateful for your consideration in adjusting your evaluation score accordingly.

We sincerely apologize for the typographical error in our response to "On the Significance of Accelerating Dataset Generation (W1)"

The last row of the table is misplaced. The correct version should be as follows

This was an oversight on our part during the format conversion process. We deeply regret any confusion this may have caused.