Enhancing Consistency of Flow-Based Image Editing through Kalman Control

We greatly appreciate your constructive comments. Below, we provide our responses to the concerns raised.

[W1] Motivation for the two-stage design

• This design is motivated by the observation that previous controller-based methods have difficulty preserving editing consistency. And we find that harnessing history trajectory could mitigate this issue, which naturally aligns with Kalman control. Therefore, we propose the two-stage method: stage 1 corresponds to the controller-based method that collects the history trajectory, while stage 2 corresponds to Kalman control utilizing that history trajectory. The two-stage approach better utilizes the structural information in the trajectory and enhances the editing consistency.

[Q1] Necessity of the two-stage design

• The necessity of the two-stage design arises from the editing dilemma as presented in Figure 1. Figure 1.a shows that previous controller-based methods tend to reach under-editing or over-editing area and lead to editing failure. With the two-stage approach, we can better utilize the structural information in the history trajectory with the help of the Kalman filter and succeed in editing, as shown in Figure 1.b. Moreover, we have conducted the ablation study comparing vanilla controller-based methods and our method with Kalman filter added. Kalman filter significantly improves the performance of the controller-based method, demonstrating the effectiveness of our two-stage design. Please refer to Table 5 in the Experiment section for more details.

[Q2] Time cost comparison

• We have presented the running time comparison of different methods. Our proposed Kalman-Edit* is slower than RF-Inversion and Flowchef, but faster than all other baselines. This shows that our approach strikes a competitive balance between editing performance and efficiency. Please refer to Figure 4 in the paper for more details.

[W2/Q3/Q4] Quantitative comparison with more recent diffusion-based methods

• We conduct additional experiments on the three listed diffusion-based methods. The results are shown in the two tables below. Our method still shows better performance across most metrics on SFHQ and HQ datasets compared to more advanced inversion schemes Direct Inversion [1], NMG [2], and SPDInv [3]. This shows the effectiveness of our approach.

Quantitative comparison on the SFHQ dataset

Quantitative comparison on the HQ dataset

---

[1] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. Direct inversion: Boosting diffusion-based editing with 3 lines of code. ICLR 2024.

[2] Cho, H., Lee, J., Kim, S. B., Oh, T. H., & Jeong, Y. (2024). Noise map guidance: Inversion with spatial context for real image editing. ICLR 2024.

[3] Li, R., Li, R., Guo, S., & Zhang, L. (2024, September). Source prompt disentangled inversion for boosting image editability with diffusion models. ECCV 2024.

We are grateful for your thoughtful suggestions and respectfully address your primary concerns below:

[W1/W2] Lacking diffusion-based qualitative comparison and image stylization results

• We will add an additional visual comparison between diffusion-based methods and ours. Since we can not upload the visualization result here, we conduct an additional quantitative experiment on the PIE benchmark [1] to show the effectiveness of our method in the image stylization task. Our method owns superior performance on the image stylization task across different metrics, compared to both diffusion-based and flow-based methods.

[W3] Concern about editing quality

• For Figure 2, our approach achieves good editing quality and maintains strong structural consistency. To verify this, we conduct an additional experiment to evaluate the local area similarity. As shown in the table below, our approach achieves the best similarity results across all evaluation metrics. This demonstrates the strong structural consistency of our method. Some of the changes in Figure 2 may appear minor because the prompt specifically targets modifications in a small local area. To better illustrate the results and demonstrate the effectiveness of our method, we will add more high-resolution image results to the Appendix.
• For Figure 3, we would like to highlight that our approach preserves more face structural consistency and has higher face similarity. We qualitatively evaluate the average Face Rec metric of the images in Figure 3. Our results have 0.4021 average Face Rec, which is lower than RF-Inversion with 0.4213 average Face Rec. This shows that our method preserves better facial consistency.
• For Figures 5 and 6, all the images are from the HQ dataset and edited with long and complex prompts. In both Figures, we only sketch the main editing area, which does not reflect the whole target prompt. We are sorry about the confusion. For example, the teddy bear case is edited with the target prompt “A teenager in a grey hoodie and blue jeans is sitting on a yellow chair at a wooden desk studying from a book and using a laptop. The room has blue walls, a shelf with books and a clock, a large framed space poster on the wall, and a teddy bear on the floor next to a backpack and scattered school supplies.” Therefore, the teddy bear appears in the target prompt and should be included in the edited image as expected. We will add the whole target prompts of each editing case to the Appendix.
• For Figure 9, most cases can reflect the effectiveness of our approach. Regarding the case of the mentioned cup, since the editing area is very small and the image is in low resolution, it does not show up effectively. We sincerely thank you for the reminder and would add high-resolution cases here.

[W4] Lacking high-resolution images

• We will add higher-resolution images to our paper. Thank you for your valuable advice.

[Q1] Lacking ablation study results

• We have conducted the ablation study between the previous one-stage controller-based method (i.e. w./o. Kalman filter) and our two-stage approach (i.e. w./ Kalman filter) in our paper. The ablation results demonstrate the effectiveness of our approach. Please refer to Table 5 for more details and Section 4.2 for more analysis.

[Q2] Concern about extension to video editing

• Our work focuses on the image editing task. The effectiveness of our approach for video editing remains to be explored. Compared to RF-Edit, our method does not require the modification of attention values and is expected to have better video editing flexibility and quality. We would try to extend our method to video editing in future work.

Quantitative comparison on the PIE benchmark image stylization task

Quantitative evaluation of local area similarity in Figure 2

---

[1] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. (2024). PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code. ICLR 2024.

Thanks to the authors for the additional evaluations and thoughtful responses. However, concerns regarding W3 and qualitative results remain unresolved.

The claim that the model shows strong structural consistency may not be convincing. In many examples, only small parts of the image are edited, or only textures are changed. These cases naturally lead to high consistency, without proving the model’s robustness. In Figure 2, most edits are minor or limited to small regions, which supports this concern.

In Figure 5, the issue raised about the top-left image (changing child to boy) was said to be a misunderstanding due to the missing full prompt. However, even with the complete prompt—“A teenager in a grey hoodie and blue jeans is sitting on a yellow chair”—the output does not match the description. The boy is not sitting on a chair and is not wearing a grey hoodie. The claimed cartoon-style transformation is also unclear. The before-and-after images look almost the same in terms of style.

The concern regarding Figure 6 was previously mentioned in the earlier review and remains unresolved. In particular, the cat and parrot images are still not properly reflected target prompt (Egyptian style, snowy landscape, ...) in the edited results, which further reinforces the concern.

In image editing, qualitative results are just as important as quantitative scores. Although the paper reports good quantitative performance, the qualitative results are not convincing. Most edits only add or change small objects or textures, and the images are shown at low resolution. As a result, it is difficult to clearly evaluate the method’s robustness from the figures provided.

Thanks to the authors for the detailed responses.

However, my concern that most edits are minor or confined to small regions remains unresolved. While the top-right example in Figure 5 indeed shows a large-region edit as claimed, the majority of examples in the main paper and the supplementary material continue to illustrate small, local edits. Regarding the qualitative comparisons on the ZONE and DIV2K datasets, the evaluation covers only two baselines and four scenes (Figure 2), which is too limited to support broad claims. For Figures 5 and 6, the authors state that the baseline models failed; however, no side-by-side qualitative comparisons are provided. Consequently, it is not possible for a reviewer to verify these claims. In Figure 5, the title of the figure describes a style-transfer example, but the bottom-left image appears to add an Egyptian mural to the background rather than transforming the entire image into an Egyptian style, which creates confusion between the description and the actual result.

Even if space constraints prevented extensive comparisons in the main paper, the additional results section—where space is typically less constrained—should have included more comprehensive qualitative comparisons. Moreover, many visual results are uploaded at low resolution; in some cases (e.g., Figure 9, “adding cup”), the images are degraded to the point that it is difficult to judge the edits. Since the central task is image editing, low-resolution visuals hinder a fair assessment and are not recommended.

Taking these points together, my concerns about the qualitative results remain unresolved. Substantial revisions would be necessary to address them, and in the paper’s current state, I cannot recommend acceptance. I regrettably recommend rejection, as I believe this needs another round of reviewing, where reviewers can inspect the visual comparison with clear and more qualitative results.

Thank you for your constructive feedback.

Due to this year’s review policy, we are not permitted to upload additional qualitative results at this stage. We have already provided extensive quantitative comparisons demonstrating our model’s superior performance. We have also included many qualitative results, including the comparisons in Figures 2 and 3 in the main paper, large area modification cases in Figures 5 and 6, additional comparison cases in Figure 7, as well as real-world editing cases in Figures 8 and 9. In addition, we would love to include more comprehensive qualitative comparisons and higher-resolution images in the revision. Specifically, we will:

• Provide visualization comparisons for each case in Figures 5 and 6 in the Appendix, highlighting our model’s superior editing capabilities.
• Add more editing cases involving large-area modifications, similar to those in Figures 5 and 6, and compare them against all other baselines.
• Upload higher-resolution results for each case in Figure 9, along with additional editing cases using a broader variety of prompt types.

We appreciate your valuable feedback again and hope this helps address your concerns.

We sincerely appreciate your valuable feedback. Our response is as follows.

[W/Q] Reason of Kalman-Edit* owning higher CLIP-T performance

• Kalman-Edit* achieves significantly higher performance on CLIP-T in Table 3, primarily because the evaluation datasets, Zone and DIV2K, are dominated by editing tasks involving local modifications like object addition or replacement. Though the second-pass inversion utilizes more structural information, it is more constrained and lacks flexibility when tackling small area modifications.
• By contrast, the shortcut estimation in Kalman-Edit* introduces greater flexibility in adhering to the target prompt as it provides a looser estimation method that utilizes a first-order approximation in the vector field. When handling the local modifications task, this method would yield a relatively small estimation error. To verify this point, we conduct an additional quantitative experiment showing the local area similarity. As shown in the table below, our approach achieves superior performance across all evaluation metrics, showing that our approach has relatively minor errors in estimating the local area in different images. This leads to better target prompt adherence, as many prompts in the evaluation datasets contain local area descriptions. As a result, Kalman-Edit* demonstrates superior performance on the CLIP-T metric.

Quantitative evaluation of local area similarity on Zone dataset

Thank you for your valuable feedback. Below, we provide our detailed responses to the concerns you raised.

[W1/Q] The sensitivity of our method to parameters

• The term "sensitive" in our method refers to the introduction of additional Kalman control parameters compared to previous controller-based approaches. In practice, as long as a few key parameters, such as the added timestep and filter strength, are properly set, the overall editing performance remains strong.
• To better show the sensitivity of our method to parameters, we conduct additional experiments. As shown in the table below, the previous controller-based method, RF-Inversion, even shows more sensitivity to strength and added steps. As these two parameters vary, the values produced by RF-Inversion show greater fluctuations compared to those generated by our approach. This suggests that the sensitivity of our method to parameter changes is lower than that of RF-Inversion.

CLIP-I Score Ablation Comparison on Zone Dataset

[W2] Difficulty of Kalman-Edit* handling complex editing scenarios

• Kalman-Edit* indeed may not become practical in some editing scenarios. But it turns out to be efficient in most editing tasks, such as style transfer and scene editing (Please see the Appendix for more visualization results).