Exploring Invariance in Images through One-way Wave Equations

We are grateful for the reviewer's valuable feedback.

$\color{blue}{[Question-1]:}$ Can you provide more detailed theoretical insights or formal proofs that justify the assumption that images naturally conform to a one-way wave equation in the latent feature space?

We acknowledge the challenge of providing a rigorous theoretical proof for this assumption. Therefore, our initial approach has been to establish strong empirical support. The successful reconstruction achieved by FINOLA, relying solely on local structure (first-order and linear after normalization), serves as compelling empirical evidence.

Furthermore, we have empirically validated key mathematical properties associated with the model. Specifically, across multiple training runs, we have confirmed:

• The invertibility of matrices $\bf{A}$ and $\bf{B}$.
• The diagonalizability of $\mathbf{AB}^{-1}$.

We believe that these empirical observations warrant further investigation and encourage the community to pursue theoretical proofs that can provide a deeper understanding of the underlying principles.

$\color{blue}{[Question-2]:}$

Beyond PSNR, have you evaluated the model using perceptually motivated metrics (e.g., SSIM, FID, or IS)?

This question is also addressed in our response to Reviewer tmjQ's Question 2. Please refer to that response for a detailed discussion of FID results.

To summarize, while FINOLA demonstrates strong performance in terms of PSNR, its results are less competitive with respect to FID. We attribute this discrepancy to our intentional use of the $L_2$ loss function, which was prioritized for its simplicity to emphasize that FINOLA's effectiveness does not rely on complex reconstruction loss formulations. We acknowledge that incorporating perceptual loss functions and Generative Adversarial Network (GAN) losses, as employed by models such as VQGAN, ViT_VQGAN, and Stable Diffusion, could potentially enhance FID scores.

$\color{blue}{[Question-3]:}$

Is it possible to report the wave speed itself? Have you considered a framework where the wave speed varies spatially or by image rather than being invariant?

The magnitude of the wave speed (represented by the eigenvalues of $\mathbf{AB}^{-1}$) for dimension 512 is provided at https://tinyurl.com/2r4s8xs3. This speed value ranges from 0.5 to 1.4.

We appreciate the reviewer's suggestion. While we acknowledge that reconstruction quality could potentially be enhanced by allowing the wave speed to vary spatially or across different images, this is beyond the scope of the present study. Our current investigation focuses on the scenario where wave speed remains invariant across spatial positions and images, with image variations limited to the initial condition (or compressed representation $\bf{q}$). Exploring the incorporation of spatially or image-dependent wave speeds represents a promising avenue for future research.

$\color{blue}{[Question-4]:}$

Given that the derivation relies on assumptions such as the invertibility and diagonalizability of weight matrices, how sensitive is the model’s performance to these assumptions? Specifically, have you examined the condition numbers of matrices B or Q in practice, and what impact do they have on the stability and performance of the model?

While the derivation relies on the assumptions of matrix invertibility and diagonalizability, these are not explicitly enforced during model training but are validated through post-training analysis. The derivations offer a mathematical interpretation of FINOLA but do not directly affect model performance.

Empirical Validation: We have empirically validated the invertibility of matrices $\bf{A}$ and $\bf{B}$ across multiple training runs. Furthermore, we have confirmed the diagonalizability of $\mathbf{AB}^{-1}$.

Condition Number: The eigenvalue spectra of matrices $\bf{A}$ and $\bf{B}$, related to their condition numbers (92 and 151, respectively), are available at https://tinyurl.com/3xz2e5am.

Sensitivity Analysis: Despite the large condition number, FINOLA is stable for the perturbations in the compressed representation space ($\bf{q}$). To evaluate perturbation sensitivity, we performed a controlled experiment where the compressed representation ($\bf{q}$) was perturbed. Perturbed representations (\bf{q}$$_p) were generated by linearly interpolating between the image representation (\bf{q}$$_i) and and a random noise vector (\bf{q}$$_n):

$\mathbf{q}_p=(1-p)\mathbf{q}_i+p\mathbf{q}_n$

The perturbation level, $p$, ranged from 0 (no perturbation) to 1.0 (full perturbation) in steps of 0.1. FINOLA and the decoder then processed these perturbed representations to reconstruct images.

The result images (see https://tinyurl.com/47whcsve) show that FINOLA is robust to small perturbations ($p$=0.1 or 0.2), but reconstruction quality diminishes with increasing perturbation.

We are grateful for the reviewer's valuable feedback.

$\color{blue}{[Question-1]:}$

Contradiction Between Lines 260 and 098: Does the assertion that \bf{AB}$$^{-1} is not diagonalizable conflict with the assumption that is diagonal (line 102 and Equation 5), especially when Figure~15 shows failing to decorrelate and yielding nearly identical outputs for both real and complex domains?

We appreciate the opportunity to clarify this point. The confusion between Lines 260 and 098 is resolved as follows:

• Line 098: This line describes an empirical observation: "we empirically observed that \bf{Q=AB}$$^{-1} matrices are diagonalizable after multiple trials of training." Subsequent derivations (Line 102 and Equation 5) are based on this observation.

• Line 260: This line reiterates that the observed diagonalizability of \bf{AB}$$^{-1} is a natural outcome of the training process, not an enforced constraint. Consequently, while \bf{AB}$$^{-1} is not theoretically guaranteed to be diagonalizable, it consistently exhibits this property in practice across training runs.

Furthermore, we wish to clarify the interpretation of Figure 15. The figure does not depict a failure to decorrelate. Instead, it illustrates two distinct variants of diagonalizable \bf{AB}$$^{-1}: one possessing complex eigenvalues and the other real eigenvalues. The complex eigenvalue variant demonstrates marginally superior performance (PSNR 26.1 vs. 25.1). The real eigenvalue case is achieved by constraining matrices $\bf{A}$ and $\bf{B}$ to be products of a shared real projection matrix $\bf{P}$ and distinct real diagonal matrices \bf{H}$$_A and \bf{H}$$_B, respectively (i.e., \bf{A=PH}$$_A, \bf{B=PH}$$_B). This configuration ensures that $\mathbf{AB}^{-1}=\mathbf{P(H}_A\mathbf{H}_B^{-1})\mathbf{P}^{-1}$, where $\mathbf{H}_A\mathbf{H}_B^{-1}$ is a real diagonal matrix.

$\color{blue}{[Question-2]:}$

Sensitivity to Small Perturbations (Equation 1): given the reliance on non-linear transformations where even small perturbations could significantly alter the final results?

This is an excellent point. To assess the method's sensitivity to perturbations, we conducted an experiment where perturbations were introduced into the compressed representation space. Specifically, perturbed representations (\bf{q}$$_p) were generated through linear interpolation between an image's representation (\bf{q}$$_i) and a randomly sampled noise vector (\bf{q}$$_n):

$\mathbf{q}_p=(1-p)\mathbf{q}_i+p\mathbf{q}_n$

The interpolation parameter, $p$, was varied from 0 (no perturbation) to 1.0 (full perturbation) in increments of 0.1. The perturbed vectors ($\mathbf{q}_p$) were then processed by FINOLA and the decoder to reconstruct images.

The result images, available at https://tinyurl.com/47whcsve, demonstrate that FINOLA exhibits robustness to small perturbations ($p$=0.1 or $p$=0.2). However, reconstruction quality degrades as the perturbation level increases.

$\color{blue}{[Question-3]:}$

Role and Interpretability of Matrices $\bf{A}$ and $\bf{B}$: including potential geometric or statistical interpretations, and how might elaborating on their function enhance the reader’s understanding of the proposed approach?

Thank you for this question! Understanding the meaning of matrices $\bf{A}$ and $\bf{B}$ is key to grasping FINOLA's core mechanism.

Definition: Matrices $\bf{A}$ and $\bf{B}$ model the local structure of the latent vector space \bf{z}$$(x,y), representing horizontal and vertical spatial transformations of the feature map:

• $\Delta_x\mathbf{z}(x,y)=\mathbf{A}\hat{\mathbf{z}}(x,y)=\mathbf{P}_A\mathbf{\Lambda}_A\mathbf{P}^{-1}_A\hat{\mathbf{z}}(x,y)$

• $\Delta_y\mathbf{z}(x,y)=\mathbf{B}\hat{\mathbf{z}}(x,y)=\mathbf{P}_B\mathbf{\Lambda}_B\mathbf{P}^{-1}_B\hat{\mathbf{z}}(x,y)$

where \bf{P}$$_A and \bf{P}$$_A are the eigenvector matrices, \bf{\Lambda}$$_A and \bf{\Lambda}$$_B are the corresponding diagonal eigenvalue matrices. We confirm $\bf{A}$ and $\bf{B}$ full rank and diagonalizable across multiple training runs.

Interpretation: Imagine points in the latent space connected by "springs". Matrices $\bf{A}$ and $\bf{B}$ define the stiffness of these horizontal and vertical springs, while eigenvectors (\bf{P}$$_A, \bf{P}$$_B) indicate their directions.

• Diagonalization: Diagonalizing $\bf{A}$ and $\bf{B}$ simplifies this, projecting latent space so each dimension (eigenvector) is an independent spring with stiffness (eigenvalue).

• Horizontal and Vertical Changes: The projected horizontal change ($\Delta_x\bf{z}$) and vertical change ($\Delta_y\bf{z}$) become scaling operations, with eigenvalues from \bf{\Lambda}$$_A and \bf{\Lambda}$$_B determining scaling strength along each eigenvector.

In essence, $\bf{A}$ and $\bf{B}$ encode directional "stretching/compressing" factors governing local feature map changes, with eigenvalues representing the strength of these factors.

We sincerely thank the reviewer for the thoughtful feedback and valuable suggestions, which have significantly improved the quality of our paper.

$\color{blue}{**[Question 1]:**}$

While ImageNet is a comprehensive dataset, could the authors evaluate FINOLA on other datasets (e.g., ADE20K, or medical images) to assess generalization?

Thank you for suggesting the importance of assessing generalization. To address this, we conducted a novel experiment applying FINOLA to computed tomography (CT) data.

Task: The objective of this experiment is to predict a CT image from its corresponding CT projection data. Both CT images and CT projection data are represented as 2D images.

Dataset: The CT dataset, comprised of brain CT scans, consists of 47,000 training samples (CT image and CT projection pairs) and 6,000 test samples. The images have a resolution of 256x256.

FINOLA Implementation: In this implementation, the CT projection data is initially encoded into a single vector, denoted as $\mathbf{q}_p$. Subsequently, two decoding branches are employed. The first branch utilizes FINOLA to generate a feature map from $\mathbf{q}_p$, followed by a decoder to reconstruct the CT projection data. The second branch linearly transforms $\mathbf{q}_p$ into a vector $\mathbf{q}_i$, which is then processed by FINOLA and a decoder to predict the corresponding CT image. Notably, the two decoding branches share the core processing components (FINOLA and the decoder) but operate on distinct, linearly correlated compressed representations ($\mathbf{q}_p$ and $\mathbf{q}_i$).

Result: The following table presents the results of the CT image prediction task. FINOLA outperforms the two baseline methods, demonstrating its capacity to generalize to a new task and dataset.

[1] InversionNet: An efficient and accurate data-driven full waveform inversion. IEEE Transactions on Computational Imaging, 2019.

[2] Fast and flexible x-ray tomograph using the astra toolbox. Optics Express, 2016.

$\color{blue}{**[Question 2]:**}$

The evaluation primarily relies on PSNR, which does not fully capture perceptual quality. Have the authors considered using other metrics?

The table below presents a comparison of FINOLA with the first stage (learning an autoencoder and vector quantization in the latent space) of multiple generative methods, assessed using both PSNR and Fréchet Inception Distance (FID) metrics.

While FINOLA achieves good performance in terms of PSNR, its results are less competitive with respect to FID. This divergence is a consequence of our deliberate choice to employ the $L_2$ loss function. We intentionally prioritized a straightforward loss function to emphasize that FINOLA's efficacy does not depend on complex reconstruction loss formulations. To further optimize FID scores, we acknowledge the potential benefits of incorporating perceptual loss functions and Generative Adversarial Network (GAN) losses, as demonstrated in models such as VQGAN, ViT_VQGAN, and Stable Diffusion.

It is crucial to reiterate that FINOLA's primary objective differs from that of a generative model. Our focus is not on image generation per se, but rather on introducing a novel perspective for understanding images. FINOLA transforms an image into a compressed representation ($\mathbf{q}$) that enables a remarkably simple autoregressive process for image reconstruction, relying exclusively on first-order and linear (after normalization) local relationships.

$\color{blue}{**[Question 3]:**}$

Qualitative results are reported on a limited number of images, could the authors provide results on different images?

Additional FINOLA reconstructed images, encompassing diverse categories such as natural scenes, human portraits, facial images, animal photographs, air and land transportation, and medical images, are available at https://tinyurl.com/4fdw47xm.

$\color{blue}{**[Other Comments]:**}$

Thanks for the helpful comments on writing and typos. We will address all these points in the final draft, including correcting the typos you noted, reducing the table sizes, moving the key results from the appendix, and improving the clarity of Figure 5.

We sincerely thank the reviewer for the thoughtful feedback and valuable suggestions, which have significantly improved the quality of our paper.

$\color{blue}{**[Weakness 1]:**}$

This method requires a lot of computing resources, especially when the feature map is high resolution. Therefore, it may be limited in practical applications.

While we acknowledge that generating high-resolution feature maps with FINOLA is computationally demanding, our parallel implementation enables its application in practical scenarios. As demonstrated in Figure 4, the entire process—including encoding, generating a 64x64 feature map using FINOLA, and decoding—can be completed in just 2.6 seconds on a MacBook Air equipped with an Apple M2 CPU.

Furthermore, we emphasize that a key contribution of this work lies in offering a novel perspective on understanding images. FINOLA transforms an image into a compressed representation ($\mathbf{q}$) that facilitates a remarkably simple autoregressive process for image reconstruction, relying solely on local relationships that are first-order and linear after normalization.

In essence, the FINOLA encoder learns a compressed representation that effectively discards explicit spatial information while preserving essential information about the image's inherent local relationships. This allows FINOLA to reconstruct the image by exploiting these consistent local structures.

$\color{blue}{**[Weakness 2]:**}$

This method is only applicable to a specific type of image, that is, images with linear structures. For other types of images, this method may not be applicable.

Empirically, we have demonstrated the versatility of this method across a wide range of image types. Additional FINOLA reconstructed images, encompassing multiple categories such as natural scenes, human portraits, facial images, animal photographs, air and land transportation, and medical images, are available at https://tinyurl.com/4fdw47xm.

Furthermore, our approach exhibits good performance with scientific images, such as seismic data and tomographic (CT) scans. For instance, in the domain of CT imaging (predicting a CT image from corresponding X-ray projection data), FINOLA not only performs well for both modalities (CT projection data and CT images) but also reveals a notable finding: the corresponding pairs of CT projection data and CT images exhibit a simple linear relationship between their compressed representations ($\mathbf{q}$). Please see more details in our reply to reviewer tmjQ (Question 1).

$\color{blue}{**[Weakness 3]:**}$

The explanation in the article is not detailed enough, for example, it does not go into details about how to choose the initial conditions and how to train the model. These details may affect the actual effect of this method.

Thanks for the suggestion. We will add more training details. The initial condition (or compressed vector $\mathbf{q}$) is generated by the encoder. The encoder, FINOLA parameters (matrices $\mathbf{A}$ and $\mathbf{B}$), and decoder are learned end to end by using $L_2$ loss between the reconstructed and original images.

Thank you for pointing out the need for more detailed explanations. Most of details are included in Appendix B.2. We will ensure clear cross-referencing between the main paper and Appendix B.2.

To clarify the points you raised:

• Initial Condition: The initial condition, represented by the compressed vector q, is generated by the encoder network. This network maps the input image to the compressed representation.

• Training: The model is trained end-to-end using backpropagation. The encoder, FINOLA parameters (matrices $\mathbf{A}$ and $\mathbf{B}$), and the decoder network are jointly optimized to minimize the $L_2$ loss between the reconstructed and original images. Additional details are listed in Appendix B.2.