On the Hidden Waves of Image

Thank you for dedicating your time and effort to provide feedback on our work. Below, we answer the questions that have been raised.

---

[Weakness 1] Experimental design: Drawing conclusions from the current results is uncertain due to the limited number of data points and the lack of significant differences. For instance, the interpretation of Table 3 remains unclear. Should we consider choosing initial condition positions different from the center? Overall, experiments were conducted from various perspectives, they do not yield meaningful insights or conclusions.

Thanks for this constructive suggestion. We will reorganize the experiment section and showcase two key conclusions and three main ablations as follows:

Conclusion I: Hidden-Wave Enhances Mathematical Decription. Hidden-Wave demonstrates a superior mathematical description by employing one-way wave equations, resulting in more accurate reconstructions compared to the two PDEs in FINOLA. Figure 4 in the paper supports this conclusion.

Conclusion II: Hidden-Wave Outperforms Conventional Methods.

Comparison with discrete cosine transform (DCT): The table below showcases the superiority of Hidden-Wave over DCT. DCT operates per 8x8 image block, retaining the top-left $K$ coefficients (in zig-zag manner), with the rest set to zero. Compared to DCT with various $K$ values (1, 3, 6, 10), Hidden-Wave consistently achieves higher PSNR with a smaller latent size.

Notably, in the last row, the term "mix" denotes placing 8 initial conditions at different positions, as opposed to overlapping at the center.

Comparison with discrete wavelet transform (DWT): We compare Hidden-Wave with DWT in the table below. Three scales are chosen for wavelet decomposition. The comparisons are organized into three groups: (a) using only the LL subband at the coarsest scale (scale 3), (b) using all subbands (LL, LH, HL, HH) at the coarsest level, and (c) using all subbands at the finer scale (scale 2). Hidden-Wave achieves higher PSNR with a smaller latent size.

Ablation I: Hidden-Wave Consistency across Feature resolutions and Image Sizes. Table 1 in the paper supports the claim that Hidden-Wave consistently performs well across multiple feature resolutions and image sizes.

Ablation II: Complex-valued Wave Speeds Improve Accuracy. Table 2 in the paper indicates that complex-valued wave speeds yield better accuracy compared to real-valued wave speeds.

Ablation III: Scattering Initial Conditions Enhance Performance. The table below demonstrates that scattering initial condition positions on a circle consistently outperforms overlapping at the center.

This reorganization highlights the key conclusions and ablations, providing a clearer and more structured presentation of the experimental findings.

---

[Weakness 2]: The phenomenon is undoubtedly intriguing and interesting, it remains unclear how this method can be applied to future research or practical applications. The paper would be improved by including discussions on potential future perspectives.

Excellent point! We will include discussions on potential future perspectives, encompassing (a) understanding the relationship between multiple initial conditions, (b) exploring the distribution of real/noise images in the latent space of Hidden-Wave, (c) investigating the application of Hidden-Wave in self-supervised learning, and (d) exploring the potential use of Hidden-Wave in image compression.

Thank you for dedicating your time and effort to provide feedback on our work. Below, we answer the questions that have been raised.

---

[Weakness 1]: Contribution: It is stated in section 2, that the prior work FINOLA is extended in two ways: a) generalizing FINOLA to a set of one-way wave equations and b) relaxing the local constraints. Yet, a) is merely an interpretation, and b) is done by summing over multiple FINOLA solutions. I consider this to be rather simple with the increase in PSNR being very natural due to the larger latent space of the resulting model.

Thank you for this comment. Let us clarify the contribution of Hidden-Wave over FINOLA as follows:

I. Improved Mathematical Description: Hidden-Wave (one-way wave equations) enhances the mathematical description of images by generalizing FINOLA. Unlike FINOLA, which connects the partial derivatives of the feature map $\mathbfit{z}(x, y)$ along the $x$ and $y$ axes using the current feature values, Hidden-Wave directly links these derivatives through one-way wave equations:

This generalization is NOT merely interpretative. We further demonstrate that a superior solution of wave equations can be achieved by summing multiple special solutions of FINOLA, validating that, for the same feature map size, one-way wave equations offer a more accurate reconstruction than the two PDEs in FINOLA. This core insight underscores the paper's contribution:

"Mathematically, the feature map governed by one-way wave equations is more accurate than the one governed by the two PDEs in FINOLA."

II. Non-Trivial Summation over Multiple FINOLA: Summing over multiple FINOLA provides a simple yet elegant empirical validation of the generalized wave equations' optimality compared to the two PDEs in FINOLA. The feature map after summation stays within the solution space of wave equations through linearity, but departs from the solution space of FINOLA.

III. Larger Latent Space: This observation is invaluable! It sheds light on why multiple FINOLA paths effortlessly navigate the solution space of wave equations for a more optimal solution. Encoding more initial conditions simplifies the approach toward the optimal solution within the wave equation space.

Moreover, multiple FINOLA paths provide a parameter-efficient means to adjust compression rate and reconstruction quality. In contrast to vanilla FINOLA, which achieves increased reconstruction quality by enlarging the number of channels and the sizes of matrices $\mathbfit{A}$ and $\mathbfit{B}$, multiple FINOLA achieves this by adding paths (or initial conditions) for the same number of wave equations and parameters in matrices $\mathbfit{A}$ and $\mathbfit{B}$. The table below compares Hidden-Wave with FINOLA across multiple latent sizes, ranging from 512 to 4096. Both methods achieve higher PSNR by increasing the latent size differently (by increasing paths in Hidden-Wave vs. increasing channels in FINOLA). Although Hidden-Wave is slightly behind FINOLA in terms of PSNR, it maintains a constant size for matrices $\mathbfit{A}$ and $\mathbfit{B}$, which is 256 times smaller than FINOLA at latent size 4096.

Dear Reviewer Zqsi,

We are pleased to inform you that we have submitted the revised version of our paper as promised in our previous communication. In this revision, we have extensively reworked the experimental section, incorporating all experiments and discussions addressed during the rebuttal. Additionally, we have provided further clarification on the diagonalization of $\mathbfit{A}\mathbfit{B}^{-1}$ in Section 2.4. Furthermore, we have included visualizations and a comprehensive discussion on future work in the appendix.

We greatly appreciate the invaluable feedback you provided, which has been instrumental in enhancing the quality of our work.

Sincerely,

-authors

Thank you for dedicating your time and effort to provide feedback on our work. Below, we answer the questions that have been raised.

---

[Q1]: In your model, A,B matrices defined in eq 2 are real-valued matrices and learnable. How do you guarantee that AB^-1 is diagonalizable such that the eigenvalues are all real-valued? (the Lambda diagonal matrix in eq 4 contains only real values along diagonal). In general, one could only assume that AB^-1 is diagonalizable such that V and Lambda are complex-valued (if AB^-1 is not symmetric).

Thank you for your valuable feedback. Allow us to provide further clarification regarding the diagonalization process. In equation 4, it's crucial to note that we do NOT impose any constraints on the eigenvalues, allowing them to be complex-valued in the diagonalization of $\mathbfit{A}\mathbfit{B}^{-1}$. In other words, the wave speed can take complex values. It's important to emphasize that $\mathbfit{\Lambda}$ and $\mathbfit{V}$ are Not explicitly trainable; instead, they are computed from the trainable matrices $\mathbfit{A}$ and $\mathbfit{B}$ through diagonalization post-training. We meticulously examined the eigenvalues in $\mathbfit{\Lambda}$ and eigenvectors in $\mathbfit{V}$ across multiple models after training, confirming their complex nature.

In Section 2.4 of the paper, we also discuss an alternative approach wherein we constrain $\mathbfit{\Lambda}$ and $\mathbfit{V}$ to be real-valued by constraining matrices $\mathbfit{A}$ and $\mathbfit{B}$ as follows:

$\mathbfit{A}=\mathbfit{P}\mathbfit{H}_x, \quad \mathbfit{B}=\mathbfit{P}\mathbfit{H}_y$

where $\mathbfit{P}$ is the shared projection matrix for $\mathbfit{A}$ and $\mathbfit{B}$, and $\mathbfit{H}_x$ and $\mathbfit{H}_y$ are two real-valued diagonal matrices that are learnable. Consequently, the wave speed $\mathbfit{\Lambda} = \mathbfit{H}_x\mathbfit{H}_y^{-1}$ is also real-valued.

However, it's noteworthy that enforcing real values for the wave speed leads to a performance drop in reconstruction quality. Table 2 (or the table below) compares the reconstruction PSNR between complex and real speeds, showing a slight decrease from 26.1 to 24.9.

---

[Q2]: It is not very clear in Fig 1 which part is trainable. Do you also train the encoder and decoder to recover input images ? If the encoder is not trained, how is it defined? Also what is your training loss?

Trainable Components: The trainable elements in our architecture encompass both the encoder and decoder. The matrices $\mathbfit{A}$ and $\mathbfit{B}$ for the multi-path FINOLA are also trainable. Throughout training, the entire network is optimized end-to-end. It's essential to note that the wave speed $\mathbfit{\Lambda}$ and the coefficient matrices $\mathbfit{H}_x$ and $\mathbfit{H}_y$ are not explicitly trainable. Instead, they are computed post-training through the diagonalization process using the trainable matrices $\mathbfit{A}$ and $\mathbfit{B}$.

Training of Encoder and Decoder Indeed, both the encoder and decoder undergo training to facilitate the reconstruction of input images.

Training Loss: The training process employs mean square error over image pixels as the loss function, ensuring the end-to-end optimization of the entire network.

[Second Round Q1]: why $\mathbfit{\zeta}(x,y) \in \mathbb{R}^C$ (defined on top of page 4)

Thank you for pointing out the notation discrepancy and expressing concerns about the mathematical rigor in our paper.

The notation stating $\mathbfit{\zeta}(x, y) \in \mathbb{R}^C$ on top of page 4 was an oversight. We acknowledge this error and wish to clarify that $\mathbfit{\zeta}(x, y)$ is correctly defined as a complex-valued vector, denoted as $\mathbfit{\zeta}(x, y) \in \mathbb{C}^C$. This vector results from a linear projection of the real-valued feature map $\mathbfit{z}(x, y)$ using the inverse of the complex-valued eigenvalue matrix $\mathbfit{V}^{-1}$, as expressed by $\mathbfit{\zeta} = \mathbfit{V}^{-1}\mathbfit{z}$. Notably, $\mathbfit{V}^{-1}$ is itself a complex-valued matrix, as indicated by the relationship $\mathbfit{A}\mathbfit{B}^{-1} = \mathbfit{V}\mathbfit{\Lambda}\mathbfit{V}^{-1}$.

We apologize for any confusion caused by this oversight and assure you that we are taking steps to enhance the clarity and accuracy of the mathematical expressions in our paper. If you have further questions or specific areas that require clarification, please feel free to let us know. Your feedback is crucial in improving the precision of our work.

---

[Second Round Q2]: I do not know how to motivate the study of real-valued eigenvalues lambda_k in Table 2, as you are assuming that A B^-1 is diagonalizable in your model. It is not clear to me whether this assumption still holds when you consider real-valued eigenvalues.

Motivation for Studying Real-Valued Eigenvalues: The motivation behind our study of real-valued eigenvalues, specifically $\lambda_k$ in Table 2, stems from the question "how the number type (real or complex) of wave speeds impacts reconstruction accuracy?" By default, wave speeds are complex-valued, as previously discussed. To explore this, we conducted this ablation study, transitioning the number type from complex to real. The results of this investigation revealed a slight degradation in performance.

Diagonalizable Assumption for Real-Valued Eigenvalues: Addressing your concern about the assumption of diagonalizability when considering real-valued eigenvalues, we affirm that the diagonalizable assumption holds in our model. We constrain the FINOLA matrices $\mathbfit{A}$ and $\mathbfit{B}$ as follows:

$\mathbfit{A}=\mathbfit{P}\mathbfit{H}_x, \quad \mathbfit{B}=\mathbfit{P}\mathbfit{H}_y$

where $\mathbfit{P}$ is a real-valued matrix, and $\mathbfit{H}_x$ and $\mathbfit{H}_y$ are real-valued diagonal matrices, all of which are learnable. This results in $\mathbfit{A}\mathbfit{B}^{-1} = \mathbfit{P}\mathbfit{H}_x\mathbfit{H}_y^{-1}\mathbfit{P}^{-1}$. Importantly, the eigenvalues $\mathbfit{\Lambda} = \mathbfit{H}_x\mathbfit{H}_y^{-1}$ and eigenvector $\mathbfit{P}$ are explicitly real-valued and learnable.

We want to highlight that the only assumption in this context is that the projection matrix $\mathbfit{P}$ is invertible or full rank. Our experiments confirm that the learned $\mathbfit{P}$ matrix is indeed invertible.

Thank you for dedicating your time and effort to provide feedback on our work. Below, we answer the questions that have been raised.

---

[Weakness 1]: In figure 4, if there is plateau for the reconstruction quality in terms of M & C

Thanks for this insightful suggestion! We conducted additional experiments to explore the impact of increasing the number of special solutions (or the number of FINOLA paths $M$) to 16. We evaluated for three choices of the number of wave equations (or the number of channels $C$), specifically 128, 256, and 512. The PSNR results are presented in the last column of the table below. In comparison to the rate of change observed over smaller $M$ values (1→2, 2→4, 4→8), the PSNR increase from $M=8$ to $M=16$ slows down, indicating the onset of a plateau.

Table: Reconstruction PSNR for different number of channels ($C$) and special solutions ($M$)