OpenWaves: A Large-Scale Anatomically Realistic Ultrasound-CT Dataset for Benchmarking Neural Wave Equation Solvers

We sincerely thank the reviewer for their constructive suggestions and are pleased that they found our dataset “valuable in terms of USCT breast-related inverse problems” and appreciated our “evaluations on multiple algorithms.” Below, we address the concerns raised:

1. Validation on real clinical data

  To validate the dataset under real-world conditions, we applied a surrogate model trained on OpenWaves to reconstruct a public USCT dataset of real breasts with tumors [1]. The imaging results, provided in the new supplementary PDF, demonstrate that models trained on simulated data can produce high-quality reconstructions of real-world data. This serves as empirical evidence that the generated phantoms closely resemble the anatomy of real organs. To further support the research community, we will include these public experimental samples in the open-source dataset, encouraging researchers to test and evaluate their own models.

2. Training and validation on different datasets

  Unfortunately, there are no other publicly available USCT datasets for training or testing. OpenWaves is the first dataset that incorporates realistic anatomy and USCT instrument settings. Prior datasets, such as OpenFWI[2] and WaveBench[3], assume unrealistic scenarios with overly simple scattering media (e.g. layered structures, Gaussian random fields) and confined computational domains (<40 wavenumbers). These limitations make them unsuitable for addressing realistic USCT imaging problems. However, we think we partially address this concern through experiments in Section 4.3.3, where neural operators trained on specific breast types or source locations are tested on unseen types and locations, demonstrating the model’s generalization to out-of-distribution (OOD) data. Additionally, the successful application of OpenWaves-trained models to real clinical data, as mentioned before, highlights the strong alignment between the simulated and real data distributions.

3. Insights on neural PDE designs

  Our benchmarking experiments provided multiple valuable insights into designing neural PDE solvers. For instance, multi-grid architectures, which effectively handle high-frequency oscillations, and BFNO, which incorporates multi-scattering processes in its design, outperform standard Fourier layers used in FNO for wave simulations and imaging. These findings suggest that integrating physical priors into neural PDE architectures is key to improving performance in USCT imaging. For inverse problems, we found that forward operators generalize better than direct inverse mappings trained on the same dataset. This suggests that forward simulation is inherently easier to learn, making the combination of forward neural operators with adjoint optimization a more robust and efficient solution for wave imaging. We will explicitly summarize and highlight these insights in the updated version of the paper.

We sincerely thank the reviewer again for their valuable insights and welcome further questions or discussions. Thank you!

Reference:

[1] Rehman Ali et al. 2-D Slicewise Waveform Inversion of Sound Speed and Acoustic Attenuation for Ring Array Ultrasound Tomography Based on a Block LU Solver. IEEE Transactions on Medical Imaging. 2024

[2] Chengyuan Deng et al. OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion. NeurIPS 2022.

[3] Tianlin Liu et al. WaveBench: Benchmarking Data-driven Solvers for Linear Wave Propagation PDEs. TMLR 2024.

5. Citation format

  We apologize for the oversight in citation format. This issue has been corrected in the revised version.

6. Interpretation of the dataset’s size

  The interpretation of the dataset’s size depends on its intended use. For forward simulations, each (source, phantom → wavefield) combination constitutes a data pair, resulting in 16M items. For inverse problems, as the reviewer noted, the dataset includes measurements corresponding to 8000 breast phantoms. However, as discussed in Section 4.2, training a neural surrogate for forward simulations is a more effective approach, as it captures accurate physics and supports more precise gradient-based image reconstruction. Directly learning inverse mappings often overfits to common structures in the phantoms rather than capturing the underlying scattering physics, leading to biased reconstructions. This is why we emphasize the 16M data pairs in this context, but we are happy to rephrase this for clarity.

We sincerely thank the reviewer again for the valuable suggestions and are happy to address any additional questions. Thank you!

We sincerely thank the reviewer for the thoughtful feedback and for recognizing that our dataset “could save future researchers considerable time and effort in data preparation” and that our evaluation is “relatively comprehensive.” Below, we address each of the reviewer’s concerns in detail.

1. Original contribution

  OpenWaves is the first wave PDE dataset to simulate realistic scenarios with complex scattering media and large wavenumbers, bridging the gap between existing datasets and real-world problems. It addresses a critical issue in the ML-PDE community: models over-optimized for toy settings often fail to generalize to practical challenges.

  OpenWaves also goes beyond merely using existing simulation tools: we designed a pipeline that automatically assigns accurate sound speeds to tissues, fine-tuned the hyperparameters of numerical solvers, and designed realistic USCT boundary conditions for robust USCT simulation. Additionally, the paper provides a thorough evaluation of forward and inverse neural PDE baselines, offering insights for designing effective solvers. As with prior benchmark works [1, 2], this processes of data generation, standardization, validation, and benchmarking should be recognized as significant contributions to the field.

2. Parameter selection for realistic breast generation

  We indeed carefully chose the VICTRE parameters and are happy to discuss them below.

  To ensure shape diversity and anatomical realism, we adjusted five key parameters—a1b, a1t, a2l, a2r, and a3—that control the breast’s bottom, top, left, right, and outward scales, respectively (units in cm). These were sampled from truncated Gaussian distributions: a1b, a1t, a2l, a2r$\sim\mathcal{TN}(5.0, 2.0, 3.5, 7.5)$, and a3/a1b$\sim\mathcal{TN}(1.4, 0.1, 1.0, 1.5)$, with $\sim\mathcal{TN}(\mu,\sigma,𝑎,𝑏)$ representing a truncated Gaussian distribution in the interval (a, b).

  To model the internal structure, we first adjusted the targetFatFrac parameter to control fat distribution, as it mainly determines the division of four breast types. Typically, the targetFatFrac parameter of Extremely Dense, Heterogeneous, Fibroglandular, and Fatty types is respectively in range of (0, 0.25), (0.25, 0.5), (0.5, 0.75), and (0.75, 1.0). We also fine-tuned the backFatBufferFrac parameter in range of (0, 0.01) to force a little fraction of phantom in nipple direction to be fat. To define skin properties, we mainly adjusted the following parameters: SkinScale in range of (200,400), SkinScaleNippleDir in range of (5,20), and skinStrength in range of (0.5,2.0).

  We will include these additional details about the data generation process in the Appendix in camera-ready version.

3. Quality assessment of the generated phantoms

  We agree that assessing the quality of the generated phantoms is essential for establishing the dataset as a reliable benchmark. In the newly uploaded supplementary PDF, we report a new experiment where a neural surrogate model trained on OpenWaves effectively reconstructs a public clinical dataset of human breasts with tumors[3]. These results empirically validate that OpenWaves captures key complexities of real-world samples. With medical doctors on our team, we plan to include additional assessments, such as external validation from clinical experts, in the revised manuscript.

4. Distribution of breast types

  We apologize for not clearly explaining the rationale for the breast type distribution. According to [4], the ratio of Heterogeneous, Fibroglandular, Fatty, and Extremely Dense types is [28%, 40%, 23%, 9%]. We adjusted this slightly to account for the higher breast density in Asian populations. Additionally, we found a labeling error where the counts for Heterogeneous and Extremely Dense breasts were swapped. This has been fixed in the revised version.

Reference:

[1] Chengyuan Deng et al. OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion. NeurIPS 2022.

[2] Tianlin Liu et al. WaveBench: Benchmarking Data-driven Solvers for Linear Wave Propagation PDEs. TMLR 2024.

[3] Rehman Ali et al. 2-D Slicewise Waveform Inversion of Sound Speed and Acoustic Attenuation for Ring Array Ultrasound Tomography Based on a Block LU Solver. IEEE Transactions on Medical Imaging. 2024

[4] Li F, Villa U, Park S, et al. 3-D stochastic numerical breast phantoms for enabling virtual imaging trials of ultrasound computed tomography[J]. IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 2021, 69(1): 135-146.

Dear reviewer, we hope you’ve had a chance to review our rebuttal and the additional experiments (Supplementary Materials, zip file) we provided in response to your comments. Given only five days left in the rebuttal period, if there are any remaining concerns or questions, we’d be happy to clarify or provide further results. Please let us know if our response has resolved your concerns.

Dear Reviewer,

With only one day left in the rebuttal period, we kindly and urgently request your review of our rebuttal and the additional experiments provided in the Supplementary Materials (zip file) in response to your comments. If you have any remaining concerns or questions, we would be more than happy to clarify them or provide further results as needed. However, a lack of response at this stage would significantly impact the fairness of the evaluation process for our work.

Thank you!

We sincerely thank the reviewer for their valuable feedback. We are pleased that the reviewer found our paper to “provide a comprehensive evaluation of data” with “clear presentation.” Below, we address each of the reviewer’s concerns in detail.

1. Main contribution

  OpenWaves is the first wave PDE dataset designed to simulate realistic scenarios with complex scattering media and large wavenumbers, addressing a critical gap in the ML-PDE community. Existing datasets often oversimplify problems by assuming small wavenumbers and basic scattering structures (e.g., layered geometries, Gaussian random fields), resulting in models that overfit toy settings and fail to generalize to real-world challenges. OpenWaves addresses this gap by introducing real-world-like complexity and conducting extensive evaluations of forward and inverse neural PDE baselines, providing valuable insights into solver design, as detailed in our summarized rebuttal to all reviewers.

  Regarding data generation, simulated datasets are standard in the neural PDE community and align with the practices of ICLR and other prestigious AI conferences. As with prior works [1, 2, 3], the processes of data generation, standardization, validation, and benchmarking are recognized as significant contributions to the field. Furthermore, OpenWaves also goes beyond merely using existing simulation tools: we designed a pipeline that automatically assigns accurate sound speeds to tissues, selected appropriate numerical solvers, fine-tuned their hyperparameters, and designed realistic USCT boundary conditions for robust USCT simulation.

2. Differences from existing datasets

  As discussed in Sec. 2.2 of the original paper, OpenWaves differs significantly from prior datasets like OpenFWI and WaveBench. OpenWaves provides anatomically realistic breast phantoms, real-world USCT imaging parameters, complex scattering media, and high wavenumbers. Additionally, it is more than an order of magnitude larger than these datasets, enabling studies of highly oscillatory PDEs, data efficiency, and scaling properties of neural architectures. Key differences are summarized in the following table:

3. Validation on real clinical data

  We performed a new experiment in the new supplementary PDF to validate the dataset. A surrogate model trained on OpenWaves was applied to reconstruct a public USCT dataset of real breasts with tumors [4]. The imaging results show high-quality reconstructions, demonstrating that OpenWaves-generated phantoms closely resemble the anatomy of real organs. These results will be added to the revised manuscript. We will also highlight this public dataset in our open-source release to enable researchers to test and evaluate their own models.

4. Time complexity for dataset generation

  The dataset generation involves two main stages:

1. Breast Phantom Generation: High-resolution breast phantoms using VICTRE took approximately 40 hours on an Intel i7-13700KF CPU, excluding time spent tuning hyperparameters.

2. Wave Simulation: Wavefield simulations for the entire dataset required approximately 60 hours of computation, parallelized across 8 NVIDIA A800 PCIe 80 GB GPUs and 20 CPUs. This estimation also neglects the additional time spent fine-tuning solver hyperparameters.

  We will include this information in our revised manuscript.

We sincerely thank the reviewer again for the constructive suggestions and are happy to address any further questions or discussions. Thank you!

Reference:

[1] Chengyuan Deng et al. OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion. NeurIPS 2022.

[2] Tianlin Liu et al. WaveBench: Benchmarking Data-driven Solvers for Linear Wave Propagation PDEs. TMLR 2024.

[3] Makoto Takamoto et al. PDEBench: An extensive benchmark for scientific machine learning. NeurIPS 2022.

[4] Rehman Ali et al. 2-D Slicewise Waveform Inversion of Sound Speed and Acoustic Attenuation for Ring Array Ultrasound

Tomography Based on a Block LU Solver. IEEE Transactions on Medical Imaging. 2024

We thank the reviewer for the thoughtful feedback and for recognizing our dataset as “extensive,” “anatomically realistic,” and “a significant contribution to the field.” Below we provide further explanations and additional results to address each concern raised by the reviewer.

1. Mathematical formulation

  In Section 3.1, we only define the general wave equation without specifying variable ranges. The variables’ quantitative values are explained in Section. 3.2.1 of the original paper: the region of interest (ROI) of $u(x)$ is 220mm by 220mm, $\frac{\omega}{2\pi}$ spans 300 kHz to 650 kHz, and the sources $s(x)$ are positioned on a annular ring, etc. Equation 1 presents the standard form of the Helmholtz equation, where $\nabla^2$ represents the Laplace operator$\nabla^2{u}= \frac{\partial^2{u}}{\partial{x^2}}+\frac{\partial^2{u}}{\partial{y^2}}$. Equation 2 specifies the Sommerfeld radiation condition as the standard boundary condition, ensuring the solution’s uniqueness. Here, $r = \Vert{\mathbf{x}}\Vert$ is the radial distance from the source and $n=2$ represents the spatial dimension. The partial derivative in Cartesian coordinates is defined as: $\frac{\partial{u(\mathbf{x})}}{\partial{r}}=\frac{\mathbf{x}_1}{r}\frac{\partial{u(\mathbf{x})}}{\partial{x_1}}+\frac{\mathbf{x}_2}{r}\frac{\partial{u(\mathbf{x})}}{\partial{x_2}}$. We will include additional explanations about these variables and operators in the final version to reduce ambiguity.

2. Scaling property and data efficiency

  The scaling behavior of a model depends on its expressive power. The UNet’s high loss levels indicate its inability to capture high-frequency features, reflecting a limitation in the model itself, not the dataset’s efficiency. By contrast, more expressive models like MgNO and FNO achieve lower loss values as the dataset size increases, demonstrating that OpenWaves effectively evaluates diverse baselines’ performance and scaling properties.

3. Open-sourcing the dataset

  As a dataset and benchmark paper, we will make both the dataset and baseline code publicly available upon acceptance.

4. Alignment of data distributions

  To validate the alignment between OpenWaves dataset and real-world conditions, we used a surrogate model trained on OpenWaves to reconstruct real breast USCT data[1]. The imaging results, provided in the new supplementary PDF, show high-quality breast reconstructions, demonstrating a strong consistency between the simulated and real-world data distributions.

5. Spacing of references

  We apologize for the formatting oversight in the citation spacing. This has been corrected in the revised version.

We sincerely thank the reviewer again for the valuable feedback and are happy to address any additional questions or concerns. Thank you!

Reference:

[1] Rehman Ali et al. 2-D Slicewise Waveform Inversion of Sound Speed and Acoustic Attenuation for Ring Array Ultrasound Tomography Based on a Block LU Solver. IEEE Transactions on Medical Imaging. 2024

Thank you again for your efforts in reviewing our paper. Given only five days remaining in the rebuttal period, we kindly remind you to review our response and additional experiments (Supplementary Materials, zip file), and let us know if we have satisfactorily addressed your concerns. If you have any remaining questions or require further clarification or additional results, we’d be happy to provide them.

Dear Reviewer,

With only one day left in the rebuttal period, we kindly and urgently request your review of our rebuttal and the additional experiments provided in the Supplementary Materials (zip file) in response to your comments. If you have any remaining concerns or questions, we would be more than happy to clarify them or provide further results as needed. However, a lack of response at this stage would significantly impact the fairness of the evaluation process for our work.

Thank you!