Accelerating Eigenvalue Dataset Generation via Chebyshev Subspace Filter

We sincerely thank the reviewer for their insightful and constructive feedback. Below we respond to your comments as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score and your confidence. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our work.

Summary

The method’s core contribution is in numerical linear algebra and scientific computing...

We proposes SCSF, an algorithm designed to accelerate eigenvalue dataset generation. Reducing the computational cost of dataset generation is critically important for AI advancements, particularly in scientific machine learning. Below, we highlight key evidence supporting the significance of this task:

• Motivation:
  • As discussed in our Introduction (Section 2), efficient dataset generation is a bottleneck for AI-driven scientific computing.
  • The AI4Science review ([1], Section 9.2.9 and Section 7.5.2) notes: "A primary limitation of learned solvers is the requirement of sufficient training data generated by expensive numerical solvers."
  • Leading ML conferences have published related work emphasizing the importance of efficient dataset generation for AI applications (e.g., ICML 2022 [2], ICLR Spotlight 2024 [3], ICML 2024 [4]).
• Key Insight: Traditional numerical linear algebra focuses on optimizing single-problem solvers, whereas dataset generation requires solving multiple related problems—a challenge unique to AI-driven scientific computing. Our work leverages problem correlations to accelerate dataset generation, designing SCSF specifically for this task.

Q1

• We clarify that SCSF does not require global spectral similarity—only that locally nearby problems (in parameter space) share spectral structure.

• This is justified because: Scientific eigenvalue problems derive from physical equations, where eigenvalues/vectors vary continuously with parameters. If parameters and spectra were uncorrelated, ML models would fail to learn the mapping—rendering dataset generation moot.

• Empirical Analysis: When spectral similarity is weak, SCSF degrades to a Chebyshev subspace iteration (equivalent to disabling filtering). We demonstrate this on the Poisson dataset; precision 1e − 12; $L = 100$; Matrix dimension 2500:

  • 	SCSF	SCSF (no sort)	SCSF (rand fliter)	Chebyshev subspace iteration
    time (s)	9.9	14.2	14.8	14.8

  • ‘rand filter’ randomly generates an invariant subspace for filtering each time the solution is solved, simulating the situation where these problems have no spectral similarity at all.

  • we can see that when there is no spectral similarity, SCSF will degenerate to Chebyshev subspace iteration. This situation is similar to SCSF without sort. This indirectly shows the importance of sorting algorithm for SCSF.

Q2

• We agree that evaluating dataset quality is crucial. Below, we provide theoretical and empirical evidence that SCSF-generated data does not compromise ML training:

1. Theoretical Guarantees: SCSF is only an acceleration algorithm, not a data augmentation algorithm. Under specified error tolerances, SCSF does not alter the generated dataset’s validity. In numerical linear algebra, "ground truth" eigenvalues are typically replaced by high-precision numerical solutions (e.g., with errors < 1e−8). Our experiments meet this threshold, which far exceeds standard ML data requirements.

2. Experimental Validation: We trained a Fourier Neural Operator to predict the smallest eigenfunction of the Helmholtz operator using two datasets (10,000 samples, rel. error < 1e−8): 1.SCSF 2. SciPy eigs. Results show identical convergence for both datasets, with training dynamics and final errors (relative L2 error) matching exactly:

   1. 	0	200	400	final
      eigs 1	0.93	0.031	0.010	0.008
      eigs 2	0.92	0.032	0.011	0.008
      SCSF 1	0.82	0.038	0.012	0.008
      SCSF 2	0.92	0.028	0.010	0.008

   2. To mitigate randomness, we repeated training twice per dataset. The results confirm that SCSF does not affect MLmodel performance.

• Conclusion: SCSF accelerates data generation without sacrificing downstream ML utility.

Q3

Please see the response to reviewer dk1n's Q2

[1] Artificial intelligence for science in quantum, atomistic, and continuum systems

[2] Lie Point Symmetry Data Augmentation for Neural PDE Solvers

[3] Accelerating data generation for neural operators via krylov subspace recycling

[4] Accelerating PDE Data Generation via Differential Operator Action in Solution Space

We sincerely thank the reviewer for their insightful and constructive feedback. Below we respond to your comments as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score and your confidence. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our work.

Q1

• The core innovations of this paper are as follows：
  • As stated in the abstract, SCSF introduces a two-stage acceleration framework: 1.Sorting: Rearranges the dataset into a sequence of strongly correlated problems. 2.Chebyshev Filtering: Exploits this correlation to accelerate eigenvalue computation. These components are mutually dependent—neither suffices alone.
  • (see Intro, Line 41 right side) the original eigenvalue data set generation algorithm solves each problem independently. SCSF is the first work to optimize dataset generation by leveraging inter-problem correlations via Chebyshev filtering.
• Relationship to ChASE:
  • Differences:
    • Algorithmic Structure: SCSF incorporates a novel Truncated FFT Sort module (Section 2.3, Line 134, left side), absent in ChASE.
    • Problem Scope: ChASE targets sequences of continuously varying eigenvalue systems (e.g., nonlinear eigenvalue problems [1]). ChASE cannot be directly used for the problem of eigenvalue dataset generation (there is a significant performance loss, see below).
  • Similarities:
    • Both exploit spectral approximations via Chebyshev filtering (Section 4.2, Line 272, left side). We explicitly cite ChASE [1] and related works.
  • Summary: SCSF is a specialized variant of Chebyshev-filtered subspace methods, optimized for dataset generation. Without sorting, SCSF reduces to ChASE-like performance.
• Experiment Comparison with ChASE:
  • Section 5.4 (Line 424, left side) tests SCSF with/without sorting. Without sorting, SCSF matches ChASE’s performance.
  • Key Results (Table 3, Line 397):
    • Sorting improves SCSF’s speed by 1.3–2.8×, reduces iterations by 5–50%. This fully demonstrates the performance gap between SCSF and ChASE in the dataset generation task.
    • Because the gap between tasks in the dataset is large, directly using the Chebyshev subspace iteration algorithm does not work well. These problems need to be sorted first and processed into a sequence with correlations before the Chebyshev filter technology can be effectively accelerated. This point is also mentioned in the abstract of [2].

Q2

• Necessity of Sorting: As shown in Section 5.4 (Line 424) and Q1, Chebyshev subspace iteration cannot directly accelerate the eigenvalue dataset generation task without significant performance degradation. The sorting module is essential to achieve speedup. Thus, complexity analysis of SCSF must account for sorting overhead.

• Computational Overhead of Sorting: Our experiments in Section 5.4 (Line 436) and Table 4 (Line 411) show that sorting a dataset of size $10^4$ using the greedy approach takes 592s—a prohibitive cost. By contrast, SCSF’s Truncated FFT Sort reduces this overhead to 1/4 of the original time.

• To further address your concern, we provide additional analysis of how sorting enhances dataset quality:

  • We measure the similarity between matrices by computing the cosine of the principal angles between their 10-dimensional invariant subspaces (spanned by the smallest 10 eigenvectors in modulus). Smaller values indicate higher similarity (one-sided distance).

  • Setup: Matrix dimension = 6400 (Helmholtz dataset), 10k samples, precision = 1e-8, 400 eigenvalues computed, parameter matrix $P$ with dimension $p=80$, and varying truncation frequencies $k$ (SCSF default $k=20$).

    • 	No sort	k = 10	k = 20	k = 30	k = 40	Greedy
      one-sided distance	0.95	0.89	0.85	0.85	0.85	0.85
      Sort time (s)	0	110	151	193	246	593
      Average solve time (s)	66.7	52.2	40.5	40.5	40.5	40.5

  • Key Findings

    • Sorting significantly increases inter-problem correlation in the dataset (explaining the performance gain).
    • The truncation parameter $k$ affects sorting time, sorting quality, and solver time. For $k \geq 20$, solver time becomes stable, showing diminishing returns. This reflects the interplay between sorting and Chebyshev iteration.

• We will include a more detailed theoretical analysis in the final version to further solidify these insights.

[1] ChASE: Chebyshev Accelerated Subspace iteration Eigensolver for sequences of Hermitian eigenvalue problem

[2] Accelerating data generation for neural operators via krylov subspace recycling, ICLR spotlight 2024

We sincerely thank the reviewer for their insightful and constructive feedback. Below we respond to your comments as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score and your confidence. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our work.

1

For example, Section 3.2 about the Chebyshev Filter (correspondingly Algorithm 1) can benefit from a more detailed explanation on why the Chebyshev Filter is useful in this case. Now, the authors only provide some formulas without further discussion. I.e. it would be useful to mention what happens with vector Cm(Y0) when m grows. Additionally, the authors should specify C0 and C1 to correctly define the recursion in (6).

• Thank you for the suggestion. Chebyshev polynomials are orthogonal function polynomials, and similar effects could theoretically be achieved with other orthogonal polynomials (e.g., Legendre). However, we chose Chebyshev polynomials due to their superior approximation properties and broad applicability.
• Here, $m$ denotes the order of the Chebyshev polynomial $C_m(t)$. Higher-order polynomials generally yield better approximation accuracy for $C_m(Y_0)$, but computational costs also increase with $m$. Thus, a trade-off is necessary—we empirically set $m = 20$ for balance.
• We apologize for the lack of clarity and will include a more detailed discussion in the final version.

2

Could you please clarify what is the parameter L used in (4)? Can it be L=n (the dimension of the problem) or L<<n?

• $L$ represents the number of target eigenvalues. For large sparse matrices derived from differential operators, $L$ is generally much smaller than $n$.
• We regret the oversight and will add a precise description in the final version.

3

Here is the list of the corresponding concerns: ...

Thank you for the detailed feedback. We will revise the algorithms accordingly in the final version. Below are clarifications:

• $Trunc_k()$ truncates the Fourier transform to retain the lowest $k$ frequencies, while $FFT()$ denotes the Fast Fourier Transform.
• $dis$ (Line 8, Algorithm 2) measures the Frobenius distance between two $P$ matrices.
• Algorithm 3, Line 3: The input is the current matrix $A$; the output is an approximation of the largest eigenvalue via a few Lanczos iterations.
• Algorithm 3, Line 12: We evaluate the relative residual of eigenpairs. Due to space constraints, this was omitted; a detailed pseudocode will be added to the appendix in the final version.
• The notation assigns the constant $m_0$ to all $m_i$. We will rephrase this to avoid ambiguity.

4.1

Now Algorithm 2 uses the 'Greedy Sorting'. Is it possible to improve its complexity further by employing a more advanced sorting algorithm?

• Yes—this is a direction for future work. We plan to combine advanced sorting techniques with our 'Truncated FFT Sorting' to further optimize efficiency.

4.2

Can the peformance of Algorithm 3 be further enhanced if it also receives the truncated FFT of the input matrix computed previously in Algorithm 2?

• No. Our work focuses on parameterized eigenvalue problems. The relationship between these parameters and matrix elements to be solved is very complex. So, in numerical linear algebra, such parameters cannot directly aid eigenvalue computation.

4.3

Instead of Lanzcos iterations in Algorithm 3, is it possible to use a simple Power method?

While possible, Lanczos is preferable for two reasons:

1. Convergence: Lanczos (a high-order Krylov method) converges faster than the first-order Power Method.
2. Efficiency: For Hermitian matrices (arising from self-adjoint operators, as noted in Line 245), Lanczos is a symmetric-optimized variant of Arnoldi iteration and incurs lower computational overhead.