On a Hidden Property in Computational Imaging

Questions 1: The class of inverse problems that can be considered is unclear. Do we have an idea of which problems involved an hidden wave phenomenon? Even an insight would be welcome. Do modalities like MRI or tomography enter in the framework?

Thank you for your excellent question. To answer this question definitively, extensive experimentation across all range of computational imaging tasks is required. Works by Chen et al. (2023a, b) demonstrate that the hidden wave phenomenon is held on ImageNet, which encompasses a wide variety of data distributions. This suggests that for single-modality autoregression, the hidden wave property is likely to exist for most image data. However, the critical question is under what circumstances two modalities can share the same wave speeds and exhibit a linear correlation in their initial conditions.

We think that a possible condition for this phenomenon is that the two modalities should represent different views of the same underlying essence, implying a strong correlation between them. Additionally, we think the presence of this phenomenon is related to whether the measurement data contains sufficient information about the target property. If the uncertainty in the data is excessively large that the inverse problem is particularly challenging to solve, the hidden wave property is less likely to hold.

In our study, we tested three tasks—FWI, CT, and EM inversion—and observed that the hidden wave phenomenon holds across all of them. These tasks involve vastly different governing equations, which suggests that the hidden wave property has broad applicability across a wide range of computational imaging problems. Since Computed Tomography is already validated within this framework, we believe that MRI, another linear inverse problem, should also fall within this framework. We plan to explore additional scenarios, including MRI and other modalities, in future work to deepen our understanding of the underlying similarities and conditions that give rise to this property.

Questions 2: The multi-path FINOLA need a better description. I don't see the "multipath" in the equations.

Apologies for the confusion. The multi-path FINOLA is an implementation method of FINOLA and is not directly tied to the main equations presented in our paper. We will include the following clarification in the text:

"The multi-path FINOLA implementation divides the initial conditions vector into multiple smaller vectors, with each vector processed independently through the FINOLA framework. All paths share the same parameters. The resulting feature maps, each corresponding to a specific solution that satisfies the necessary constraints, are then aggregated to construct the final solution."

Questions 3: ne small experiment to compare the method with classical framework (LASSO with wavelets...) would interesting to have a full idea on the effectiveness of the framework.

The classical framework you mentioned, such as LASSO with wavelets, is conceptually similar to our baseline method, SIRT, which is also a widely used classical approach. To provide additional context, we conducted experiments using LASSO with Haar wavelets. The results, shown in the table below, indicate that both SIRT and LASSO with Haar wavelets yield similar performance.

---

Thank you for dedicating your time and effort to provide feedback on our work.

Dear Reviewer Uz6Q,

We hope this message finds you well. We want to extend a friendly reminder that the author-reviewer discussion period is nearing its close. Your input and perspective have been highly valuable throughout this process. If any questions or concerns about our rebuttal arise, we're here to listen and address them in a timely manner. Your insights are crucial in refining our manuscript further.

Understanding the demands of your time, we deeply appreciate your active participation in this rebuttal discussion. Your contributions are invaluable to us, and we genuinely thank you for your dedication.

Should you have any inquiries or thoughts, please don't hesitate to reach out.

Best regards,

-Authors

Dear Reviewer Uz6Q,

Thank you for your feedback and for sharing your perspective. We greatly appreciate the time and effort you have devoted to reviewing our manuscript.

To further support our validation of the proposed property and deepen our understanding of the latent space structure, we conducted an additional experiment generating a full-resolution feature map (70 $\times$ 70) in the latent space based on our proposed property, with minimal reliance on the decoder. As shown in the following table, this configuration achieves performance consistent with other setups. This result demonstrates that the decoder, with only 0.4M parameters, is dwarfed by the complexity of the linear converter and Finola modules (0.3M + 1.1M parameters). Consequently, the output is primarily governed by the regularizations, which effectively embody the proposed properties.

While we understand your decision to maintain your score, we would like to kindly ask if you have any comments or suggestions regarding the paper. Your expertise is invaluable to us, and we are committed to improving the quality and clarity of our work. If there are specific aspects of the paper that you believe could be further refined or strengthened, we would be deeply grateful for your insights.

Thank you again for your thoughtful engagement throughout this process. We look forward to hearing any additional thoughts you may have.

Best regards,

The authors

Weaknesses: In my opinion, the paper's main contribution is to propose a new architecture, and not establish any fundamental “hidden property” in computational imaging problems. State-of-the-art deep learning methods for CT are not used for comparison, making it difficult to judge the empirical superiority of the new architecture.

We appreciate your comments and understand the concerns regarding the paper's contributions. We would like to clarify that the primary goal of this work is not to propose a new architecture or achieve state-of-the-art (SOTA) performance. Rather, the focus is to uncover and validate a hidden property in computational imaging problems.

To achieve this goal, we intentionally designed a network with strong spatial regularizations and hard constraints, prioritizing mathematical simplicity and interpretability over raw performance. These strong restrictions enable us to uncover insights and highlight the hidden property we propose. While this framework does not aim for SOTA performance, it serves as a highly regulated framework for revealing the mathematical insights that constitute the core contribution of our work. This focus distinguishes our study from traditional architecture-driven advancements.

As stated in the paper, this work empirically demonstrates the hidden property, which is revealed through the strong regularization applied within the network, rather than through theoretical derivation. Analogous to how LASSO regularization reflects the sparsity property of data, we use linear constraints and the FINOLA method to regulate the latent space in alignment with the proposed property. These constraints ensure that the hidden property emerges clearly, facilitating its validation and analysis.

We hope this clarification better aligns our intended contributions with the reviewer's expectations and demonstrates how our work advances the understanding of computational imaging problems by providing novel insights into their underlying structure.

Questions 1: The usage of non-standard terminologies in comparison with the inverse problems literature. For instance, the word “modality” is used to refer to the parameter and data spaces, which can lead to confusion.

Sorry for the confusion. We intended to adopt more machine learning terminologies to serve readers from this venue better. To address this, we will clarify our terminology in the Introduction, page 2, lines 54-56, as follows:

"These methods address inverse problems involving two modalities, referring to the reconstruction of physical properties (parameter space) from measured data (data space), a process governed by linear or nonlinear mathematical equations (Kirsch et al., 2011)."

This revision aims to bridge the terminological gap between the inverse problems literature and the machine learning audience, ensuring clarity while maintaining accessibility for readers from both domains.

Questions 2: Figure 1 caption (and other places in the introduction): The phrase “latent space” is used frequently without much explanation about what exactly it refers to.

The phrase "latent space" is a commonly used concept in machine learning, referring to a lower-dimensional space of latent variables—also known as hidden or unobserved variables. These variables are not directly observable in the physical world but are learned by machine learning algorithms to capture the essential features and underlying structures of the input data.

Questions 5: Page 2: “...typically with a bottleneck in the network, they lack a deeper understanding of these latent representations.”: I don't see any such “deeper understanding” (which in itself is somewhat vague and subjective) being uncovered in this paper either.

In this work, the term "deeper understanding" refers to uncovering and validating the hidden property in the latent space that reflects the relationships of two modalities. While this is not a theoretical derivation, we demonstrate the existence of this property empirically by applying strong spatial regularizations and hard constraints to the network. These constraints simplify the latent space representation, contributing to a novel and deeper understanding of the structure of latent spaces in deep learning models for computational imaging tasks.

Questions 6: The phrase “target property” is used in several places in the paper. Could you please explain what this means?

The phrase "target property" is a shorthand for "target physical property," referring to the physical properties or parameters that need to be reconstructed or inferred. To address potential confusion, we will clarify this in the Introduction, page 2, line 78, as follows:

"..... we show that the latent spaces of both measurement data and target physical properties (shortened as target properties) are governed ......"

Questions 7: Section 2.1: It might be good to make the descriptions of the inverse problems (FWI, CT, and EM inversion) more concise.

Thank you for your suggestion. We will revise Section 2.1 to be more concise as follows:

" Full Waveform Inversion (FWI):
FWI is a well-known method to infer subsurface acoustic velocity maps ($c(\mathbf{r})$) from seismic waveform data ($p(\mathbf{r},t)$), collected via sensors during seismic surveys. $\mathbf{r}=(x,y)$ represents spatial location coordinates, and $t$ denotes time. Each receiver logs a 1D time series signal, and the collective signals from all receivers form the waveform data. Given the source term $s(\mathbf{r}, t)$ and the Laplacian operator $\nabla^2$, the process is governed by the acoustic wave equation:

$$\nabla^2p(\mathbf{r},t) -\frac{1}{c^2(\mathbf{r})} \frac{\partial^2}{\partial t^2}p(\mathbf{r},t) = s(\mathbf{r},t).$$

In this problem, the target property $\mathbf{\psi}$ is the velocity map $c(\mathbf{r})$, and the measurement data $\mathbf{P}$ is the waveform data collected by surface sensors (i.e., $z=0$), abbreviated as $p(x, t)$.

---

Computed Tomography (CT):
CT is a vital imaging technique to reconstruct an object's internal structure ($f(x, y)$) from projection data ($p(\mathbf{d}, \mathbf{s})$) of X-rays passing through the object, where $(x, y)$ are the spatial coordinates, $\mathbf{s}=(x_s, y_s)$ is the X-ray source position, and $\mathbf{d} = (x_d, y_d)$ is the detector position. Mathematically, the projection data is the integral of the attenuation coefficient $f(x, y)$ along the line segment $L(\mathbf{s}, \mathbf{d})$ connecting the source $\mathbf{s}$ and the detector $\mathbf{d}$:

$$p(\mathbf{d}, \mathbf{s}) = \int_{L(\mathbf{s}, \mathbf{d})} f(x, y) \, ds.$$

In this problem, the target property $\mathbf{\psi}$ is the attenuation image $f(x, y)$, and the measurement data $\mathbf{P}$ is the projection data $p(\mathbf{d}, \mathbf{s})$.

---

Electromagnetic (EM) Inversion:
EM inversion focuses on recovering subsurface conductivity ($\sigma$) from electromagnetic field measurements ($\mathbf{E}$ and $\mathbf{H}$). Let $\mathbf{J}$ and $\mathbf{P}$ be the electric and magnetic sources, and $\mu_{0}$ be the magnetic permeability of free space. The governing equations are the time-harmonic Maxwell's Equations:

$$\sigma\mathbf{E} - \nabla\times\mathbf{H} = -\mathbf{J},$$ $$\nabla\times\mathbf{E} + i\omega\mu_{0}\mathbf{H} = -\mathbf{M}.$$

In this problem, the target property $\mathbf{\psi}$ is the electrical conductivity $\sigma$, and the measurement data $\mathbf{P}$ is the electric and magnetic fields $\mathbf{E}$ and $\mathbf{H}$.

"

Questions 3: Abstract: “where both modalities are governed by complex mathematical equations”: This is a rather vague statement to use in the abstract. What specific governing equations are being referred to here? What latent space is talked about here? The actual contributions or the significance of the proposed method do not come out clearly from the abstract.

Governing Equations: To clarify, the "complex mathematical equations" referenced in the abstract refer to the governing equations underlying the computational imaging tasks discussed in the paper, such as the acoustic wave equation in FWI, the linear integral equation in CT, and Maxwell's equations in EM inversion. Given the diversity of these tasks, it would be overly redundant to detail each equation in the abstract. Instead, we use general terminology to highlight their commonalities, while providing a detailed review of these equations and tasks in the main text.

Latent Space: As explained in Question 2, the latent space represents a lower-dimensional space of latent variables learned by the network to capture the essential features and underlying structures of the input data.

Contributions: Furthermore, as stated in our abstract and discussed in Question 1, the primary goal of this work is not to propose a new architecture or achieve SOTA performance. Instead, the focus is on uncovering and validating the mathematical insights of the structure of latent spaces in deep learning models for computational imaging tasks.

Questions 4: “Whether an elegant mathematical relationship exists in the latent space, akin to that in the original space?”: I don't think it is a precise, well-formulated research question. The mathematical relationship between the parameter space and the data space is determined by the specific imaging modality, whereas the relationship in the “latent space” is empirically enforced (and no such relationships are shown to exist theoretically).

Recent works (Feng et al., 2022, 2024) have demonstrated that physical properties with strong mathematical relationships (e.g., waveform data and velocity map) exhibit simpler relationships (e.g., linear) when projected into latent space with appropriate encoders. Building on this idea, our work aims to further explore and advance this understanding. Unlike prior studies that relied on complex encoder designs to generate latent spaces, our approach employs autoregression governed by a set of wave equations. This introduces much stronger regularizations on the feature map, reducing the degrees of freedom in both the encoder and decoder and resulting in a more constrained and interpretable latent space structure.

As an experimental verification paper, our focus is on empirical validation rather than theoretical derivation. The validation and ablation experiments confirm that the added regularizations do not degrade performance. Instead, they demonstrate that the latent space, described by these proposed mathematical properties, is naturally contained in the potential optimal solutions of the latent space. This suggests our approach does not force the model into an artificially flawed latent space. The key contribution of this work is the discovery and empirical validation of this latent space property. While the findings are empirical, they offer a novel perspective on the mathematical structure of latent representations in computational imaging tasks.

[1] Feng, Yinan, et al. "Auto-Linear Phenomenon in Subsurface Imaging." Forty-first International Conference on Machine Learning, 2024.

[2] Feng, Yinan, et al. "An intriguing property of geophysics inversion." Thirty-ninth International Conference on Machine Learning. PMLR, 2022.

Questions 8: Page 6: “Difference with vanilla FINOLA”: This part needs to be rewritten. Currently, it gives the impression that the proposed method is capable of handling multi-modal data for reconstruction, which it isn't.

We extend FINOLA to handle two modalities in computational imaging: measurement data reconstruction and target physical property inference simultaneously. We have revised the text to clarify that the concept of "two modalities" used here differs from the conventional meaning of multi-modal data in computer vision.

" Difference with vanilla FINOLA: Unlike vanilla FINOLA, which is designed for single-modality autoregression, our method extends to two modalities by sharing parameters across both domains. While vanilla FINOLA focuses on modeling single-domain data invariants, we adapt it for computational imaging tasks by enabling the joint representation of measurement data and target physical properties. This joint representation captures the relationship between the two domains, governed by wave equations, offering a unified framework for their relationship. It is important to note that this does not involve handling multi-modal data in the conventional sense but specifically to handle two modalities in computational imaging: measurement data reconstruction and target physical property inference. "

Questions 9: Page 7: Architecture details: Do you use the same architecture for the reconstruction network for all three inverse problems considered?

Yes, the same architecture is used for the reconstruction network across all three inverse problems. However, due to differences in the data sizes for each problem, the network size and hyperparameters are adjusted accordingly.

---

Thank you for dedicating your time and effort to provide feedback on our work.

Dear Reviewer bUtp,

We hope this message finds you well. We want to extend a friendly reminder that the author-reviewer discussion period is nearing its close. Your input and perspective have been highly valuable throughout this process. If any questions or concerns about our rebuttal arise, we're here to listen and address them in a timely manner. Your insights are crucial in refining our manuscript further.

Understanding the demands of your time, we deeply appreciate your active participation in this rebuttal discussion. Your contributions are invaluable to us, and we genuinely thank you for your dedication.

Should you have any inquiries or thoughts, please don't hesitate to reach out.

Best regards,

-Authors

The response appears self-contradictory to me. While the authors claim that "the primary goal of this work is not to propose a new architecture or achieve state-of-the-art (SOTA) performance", they mention in the very next paragraph that they "intentionally designed a network with strong spatial regularizations and hard constraints". The latter is what I understood from the paper, and that is essentially equivalent to proposing a new model for the reconstruction network (i.e., an architecture that incorporates these constraints). I am still of the opinion that the claims of the paper in the abstract (and the introduction) are stronger than the actual contributions (and mildly misleading). The claims of superior interpretability and "deep understanding" of the latent representations are also not substantiated enough.

Overall, I am of the opinion that the theoretical contributions of the paper are limited to none, while the empirical performance is also underwhelming. I would, therefore, like to keep my original evaluation score as is.

Reply to comments

On the Network Design and Its Purpose. While it is true that we intentionally designed a network with strong spatial regularizations and hard constraints, this design does not contradict our claim that the primary goal of this work is not to propose a new architecture or achieve state-of-the-art (SOTA) performance. The architecture itself is not the focus; rather, it is a means to validate the proposed property effectively.

The purpose of the architecture is to translate the proposed property from mathematical equations into practical implementation, thereby empirically validating the existence of the hidden property we proposed. The network acts as a tool to impose the constraints necessary for revealing this property and is not intended as a general-purpose reconstruction model for a state-of-the-art solution.

To further support our claim that the architecture is designed to validate the proposed property and deepen our understanding of the latent space structure, rather than to achieve SOTA, we conducted an additional experiment generating a full-resolution feature map (70 $\times$ 70) in the latent space based on our proposed property, with minimal reliance on the decoder. As shown in the following table, this configuration achieves performance consistent with other setups. This result demonstrates that the decoder, with only 0.4M parameters, is dwarfed by the complexity of the linear converter and Finola modules (0.3M + 1.1M parameters). Consequently, the output is primarily governed by the regularizations, which effectively embody the proposed properties.

Theoretical and Experimental Contributions. We acknowledge that this work does not make theoretical contributions in the traditional sense. However, the focus of this paper is on experimental verification, which we believe is equally important for advancing understanding in this domain. The strong regularizations and constraints in the network are a specific embodiment and implementation of the proposed property, allowing us to empirically demonstrate its existence. This experimental approach complements theoretical work and provides a foundation for future investigations.

Dear Reviewer bUtp,

Thank you for your feedback; we genuinely appreciate your candid perspective. We understand that you disagree with our assessment of the novelty in the paper. Could you kindly elaborate on the specific aspects you disagree with? Or do you disagree with every aspect? Your clarification would be invaluable in helping us address your concerns and improve our work.

Best regards,

The authors

Weaknesses 1: The work draws very heavily on two prior works by Chen et al. 2023 (a,b). As far as I can tell neither of these works have been accepted by peer-review venues.

Although the works by Chen et al. (2023 a, b) have not yet been accepted by peer-reviewed venues, we find their ideas to be both innovative and interesting. To ensure the reliability and validity of their methods, we reproduced their results on ImageNet before apply them. Moreover, this framework performs well in our specific problem domain. We believe that interesting and reproducible research, regardless of its publication status, can provide valuable insights and serve as a solid foundation for further exploration.

Weaknesses 2: There is no theoretical motivation for the hidden wave equations, as far as I can tell, although I did not review the cited papers.

The hidden wave equations provide FINOLA, an autoregressive method, with a better mathematical description without affecting its actual operation. In essence, FINOLA is equivalent to the formulation of the hidden wave equations. It is important to note that empirical results drove the development process—FINOLA was initially proposed and demonstrated to work effectively, which then motivated the authors to formalize its underlying principles mathematically. Thus, the theoretical description arose as a retrospective effort to better articulate the method's foundations, rather than serving as its original motivation.

Questions: Can the authors speculate about similarity between the computational imaging tasks considered that might give rise to the observed phenomenon? Or do they believe this phenomenon should existing for all computational imaging tasks?

Thank you for your excellent question. To answer this question definitively, extensive experimentation across all range of computational imaging tasks is required. Works by Chen et al. (2023a, b) demonstrate that the hidden wave phenomenon is held on ImageNet, which encompasses a wide variety of data distributions. This suggests that for single-modality autoregression, the hidden wave property is likely to exist for most image data. However, the critical question is under what circumstances two modalities can share the same wave speeds and exhibit a linear correlation in their initial conditions.

We think that a possible condition for this phenomenon is that the two modalities should represent different views of the same underlying essence, implying a strong correlation between them. Additionally, we think the presence of this phenomenon is related to whether the measurement data contains sufficient information about the target property. If the uncertainty in the data is excessively large that the inverse problem is particularly challenging to solve, the hidden wave property is less likely to hold.

In our study, we tested three tasks—FWI, CT, and EM inversion—and observed that the phenomenon holds across all of them. These tasks involve vastly different governing equations, which suggests that the hidden wave property should be applicable across a wide range of computational imaging tasks. However, whether it generalizes to all such tasks remains an open question that requires further investigation. We plan to explore more scenarios in future work to deepen our understanding of the underlying similarities and conditions that give rise to this property.

---

Thank you for dedicating your time and effort to provide feedback on our work.

Dear Reviewer pMMZ,

We hope this message finds you well. We want to extend a friendly reminder that the author-reviewer discussion period is nearing its close. Your input and perspective have been highly valuable throughout this process. If any questions or concerns about our rebuttal arise, we're here to listen and address them in a timely manner. Your insights are crucial in refining our manuscript further.

Understanding the demands of your time, we deeply appreciate your active participation in this rebuttal discussion. Your contributions are invaluable to us, and we genuinely thank you for your dedication.

Should you have any inquiries or thoughts, please don't hesitate to reach out.

Best regards,

-Authors

Dear Reviewer pMMZ,

Thank you for your feedback and for sharing your perspective. We greatly appreciate the time and effort you have devoted to reviewing our manuscript.

To further support our validation of the proposed property and deepen our understanding of the latent space structure, we conducted an additional experiment generating a full-resolution feature map (70 $\times$ 70) in the latent space based on our proposed property, with minimal reliance on the decoder. As shown in the following table, this configuration achieves performance consistent with other setups. This result demonstrates that the decoder, with only 0.4M parameters, is dwarfed by the complexity of the linear converter and Finola modules (0.3M + 1.1M parameters). Consequently, the output is primarily governed by the regularizations, which effectively embody the proposed properties.

While we understand your decision to maintain your score, we would like to kindly ask if you have any comments or suggestions regarding the paper. Your expertise is invaluable to us, and we are committed to improving the quality and clarity of our work. If there are specific aspects of the paper that you believe could be further refined or strengthened, we would be deeply grateful for your insights.

Thank you again for your thoughtful engagement throughout this process. We look forward to hearing any additional thoughts you may have.

Best regards,

The authors