Physics-aligned field reconstruction with diffusion bridge

Thanks for the positive comments! We've added additional evaluations of our method using extended metrics, see following.

Q1. Can you additional explain me, why did you use only normalized metrics?

A1:We have extended the evaluation metrics to include both MSE and L1 error metrics. The results using these metrics are included in all the tables of the revised manuscript, where we observe that they indicate similar trends across our cases. For residual errors of the equations, the 'nER' metric, as presented in the manuscript, evaluates compliance with the corresponding physical constraints. For further details, please refer to Section 4.4 (Evaluation Metrics) of the revised manuscript.

---

Q2. Also, can you please explain why you checked on flow equations (like Kolmogorov and Reaction-Diffusion equations)?

A2: We chose to focus on flow equations because, to our knowledge, the tasks we address (e.g., FI and RI) are more practical and relevant in scenarios involving flow equations. These equations often exhibit complex solutions with co-existing multi-scale structures, making them well-suited for applications such as climate data forecasting and downscaling [1]. While we would have liked to include a broader range of physical systems in our study, our focus was limited to fluid-related systems due to constraints in accessing domain knowledge and relevant datasets.

---

References:

[1] Carrassi, Alberto, et al. "Data assimilation in the geosciences: An overview of methods, issues, and perspectives." Wiley Interdisciplinary Reviews: Climate Change 9.5 (2018): e535.

Thanks for spending time comment on this work! I hope the following responses can address your concerns.

Q1. while the methodology is well chosen, I don't believe the paper presents major new methodological contributions.

A1: Indeed, we borrow the idea from existing work (I$^2$SB [1] and RLHF [2, 3]) to design the two stage process while specifically tailoring it to our problem according to the characteristics of the physical field reconstruction: 1) By incorporating physics-aligned loss after the pretraining, we alleviate the optimization issue introduced by starting from a good initialization; 2) Instead of reinforcement learning in typical RLHF methods, since the physics-aligned loss is differentiable w.r.t. its input, we directly optimize the model with gradient descent truncated through the sampling path; 3) the regularization term is not simply a KL-regularization used in RLHF methods (which leverages the samples generated from the pretrained model to constrain the finetuned distribution). Instead, we constrain the finetuned distribution with the samples from pretraining data, which is more straightforward and effective in the data-driven reconstruction task.

---

Q2. some of the techniques seems somewhat adhoc, i.e. employing early stop sampling.

A2: The motivation for employing early stop sampling stems from our observation that smaller step sizes improve sample quality due to reduced discretization error, particularly when using fewer sampling steps (e.g., 10 steps). This technique is designed to enhance efficiency by reducing the step size while maintaining a low number of sampling steps (see Fig. 5). Additionally, we conducted detailed ablation studies to validate the empirical performance of these techniques, demonstrating their efficacy (see Fig. 7 and Section C of the manuscript).

---

Q3. the experimental validation is somewhat limited being restricted to 2D problems and with fairly limited datasets.

A3: We now extended our method to a complex 3D case. The results are summarized in Table 3 and Figs. 20, 21. Please refer to Section B.1 of the revised manuscript for details.

---

References:

[1] Liu, Guan-Horng, et al. "I $^ 2$ SB: Image-to-Image Schr" odinger Bridge." arXiv preprint arXiv:2302.05872 (2023).

[2] Lee, Kimin, et al. "Aligning text-to-image models using human feedback." arXiv preprint arXiv:2302.12192 (2023).

[3] Fan, Ying, et al. "Reinforcement learning for fine-tuning text-to-image diffusion models." Advances in Neural Information Processing Systems 36 (2024).

Thanks for the positive and insightful comments from reviewer BWVW on this work! I hope the following responses can resolve the reviewer's questions.

Q1. In line 083-085: What is the relationship between $m$, $n$ and $d$?

A1: $d$ is the spatial dimension of the physical domain (e.g., $d=2$ for 2D cases). Given a domain $\Omega \subset \mathbb{R}^d$, $n$ is the dimension of the discretized sample of data $\mathbf{x}_0$ (e.g., $n=65536$ for $256\times 256$ grid). $m$ is the dimension of the observation data $\mathbf{y}$, which is degenerated from the data sample (i.e., $\mathbf{y}=\mathcal{H}(\mathbf{x}_0)+\mathbf{\epsilon}$).

---

Q2. I can't quite follow the derivation of eq. (6) from eq. (4), I end up with $f_\theta+\mathbf{x}_0$ instead.

A2: Thanks for the important reminder! The parameterization of the network should be $f_\theta=-\sigma_t \epsilon_\theta + \mathbf{x}_t$. In this case, one can deviate eq. (6) from eq. (4). We have rephrased the formula in the revised manuscript to make it clearer.

---

Q3. Line 268: Doesn't the backpropagation get truncated for all steps $i<N$? That's how I interpret both Fig. 1 and Alg. 2.

A3: There is a subscript substitution in our manuscript, resulting in the confusion. As suggested by the reviewer, we now changed the notation for time steps into $0<t_T<t_{T-1}<...<t_1=1$ throughout the paper. Please refer to Eq. (8), Fig. 1 and Alg. 2 of the revised manuscript.

---

Q4. Does finetuning use early stop sampling?

A4: In our experiments, the finetuning also uses early stop sampling.

---

Q5. Does truncating the backpropagation in the finetuning step affect the adherence to physical constraints in a time-step dependent manner?

A5: In the expanded experiments, we evaluated the impact of backpropagation truncation on adherence to physical constraints. The results demonstrate that, within a fixed number of fine-tuning iterations, truncating backpropagation can enhance adherence to physical constraints without altering the trend in a time-step-dependent manner (see Fig. 8 in the revised manuscript). For further details, please refer to Section C.2 of the revised manuscript.

---

Q6. How much padding is used for sampling and does the amount of padding influence performance? Does that interact with the number of sampling steps?

A6: According to the dimension of the observations, we use padding size of 2 for FI task while 16 for RI task to make the size of the interpolated data samples consistent. We now extended the experiments to investigate how the padding size and sampling steps affect the performance (Fig. 9 in the revised manuscript). The results indicate that increasing the padding size can improve the final performance without altering the trend with respect to the number of sampling steps. Please refer to Section C.4 in the revised manuscript for details.

---

Q7. Issues in Algorithms 2 and 3.

A7: Thanks for the important reminder! We now rephrased Algorithms 2 and 3 of the revised manuscript to make them clearer.

---

Typos/Format etc.

A: Thanks for pointing out the typos! We have corrected these typos in the revised manuscript.