FlowDrag: 3D-aware Drag-based Image Editing with Mesh-guided Deformation Vector Flow Fields

[Q1] The paper lacks comparison with DragGAN, which achieves better performance in certain cases.

[A1] We provide additional qualitative comparisons between FlowDrag and DragGAN in Fig. 14(a) (please refer to the link below). To ensure fair comparisons, we reproduced DragGAN using the official GitHub repository and employed PTI [1] for GAN inversion. Since the provided DragGAN weights are limited to the StyleGAN-Human model, our comparisons focus exclusively on human images. As shown in Fig. 14(a), DragGAN struggles to preserve the original person's identity, whereas FlowDrag maintains identity effectively and exhibits better geometric consistency.

Fig 14. https://anonymous.4open.science/r/FlowDrag-B950/Fig_14.pdf

[1] Pivotal Tuning for Latent-based Editing of Real Images (ACM TOG 2022)

[Q2] I would like to see additional results from the "content removal" or "content creation" categories of the Drag100 dataset.

[A2] We applied FlowDrag to the "content creation" and "content removal" examples from the Drag100 dataset, and present additional results in Fig. 14(b)-(c). In the "content creation" category (Fig. 14(b)), FlowDrag successfully generates natural results with fewer artifacts compared to GoodDrag. Similarly, in the "content removal" category (Fig. 14(c)), FlowDrag demonstrates editing quality that is comparable or superior to GoodDrag. Although FlowDrag primarily targets "rigid edits" to preserve object rigidity during editing (as stated in our introduction), these results confirm that our method is also effective in handling non-rigid editing scenarios.

[Q3] The two orange arrows in Fig.4 overlap with text, making the figure quite messy.

[A3] Thank you for pointing this out. We will adjust the positioning of the orange arrows in Fig.4 to avoid overlap and improve visual clarity in the final manuscript.

[Q4] There is no link in citations and references to figures and tables.

[A4] We will include hyperlinks in citations and references to figures and tables in the final manuscript to enhance readability.

[Q5] The axis metrics in Fig.2 (d) and (e) are inconsistent.

[A5] Yes, the axis metrics in Fig. 2(d) and Fig. 2(e) intentionally differ. Fig. 2(d) compares methods on the DragBench dataset using metrics such as 1-LPIPS (for image fidelity) and MD (Mean Distance, for evaluating handle-point movement accuracy). In contrast, Fig. 2(e) shows comparisons on our proposed VFD-Bench dataset, which provides ground-truth edited images from video frames, enabling evaluation using RGB-level (PSNR) and feature-space metrics (1-LPIPS and MD). Fig. 2(e) specifically plots results using PSNR and 1-LPIPS. We will explicitly clarify this distinction in the final manuscript.

[Q1] What is the value of $\lambda$ in Equation 8?

[A1] The parameter $\lambda$ in Eq. (8) represents the incremental step size at each iteration, indicating the fraction of the displacement between the handle vertex ($v_h$) and the target vertex ($v_t$). However, we discovered a minor typo in the original version of Eq. (8), and we sincerely appreciate the reviewer’s careful observation. The corrected Eq. (8) is:

$v_h^{(k+1)} = v_h^{(k)} + \lambda (v_t - v_h), \quad 0 < \lambda \le 1$

In this equation, both $v_h^{(k+1)}$ and $v_h^{(k)}$ are intermediate vertices positioned incrementally between the initial handle vertex ($v_h$) and the target vertex ($v_t$). To facilitate a clearer understanding, we provide an enhanced illustration in Fig. 15 (please refer to the link provided below), complementing Fig. 3 in the paper. This formulation implies that instead of moving the handle vertex directly from $v_h$ to $v_t$ in one step, our method progressively moves it through multiple intermediate positions. Specifically, we set $\lambda=0.2$ in our implementation, meaning the vertex moves 20% closer to its target at each iteration. For example, with $\lambda=0.2$, the handle vertex path would be as follows:

$v_h (\text{handle vertex}) = v_h^{0} → v_h^{1} → v_h^{2} → v_h^{3} → v_h^{4} → v_h^{5} = v_t (\text{target vertex})$

This progressive SR-ARAP algorithm thus prevents abrupt mesh distortions by smoothly distributing large vertex displacements across multiple intermediate steps, achieving more stable and coherent deformations. We will clearly correct Eq. (8) accordingly in the revised manuscript. We sincerely thank the reviewer for this valuable clarification.

Fig 15. https://anonymous.4open.science/r/FlowDrag-B950/Fig_15.pdf

[Q2] I am also confused with the handle-matching term in Equation 9.

[A2] The handle-matching term in Eq. (9) acts as a soft constraint, ensuring handle vertices smoothly approach their intended intermediate (and ultimately final) positions. Since the SR-ARAP algorithm primarily optimizes vertex positions by minimizing local rigidity (minimal local distortion), handle vertices may not exactly reach intermediate targets. To resolve this, the handle-matching term gently penalizes deviations from target positions:

$\beta \sum_{v_h \in \text{handles}} \Bigl\|\, v_h^{(k+1)} - v_t \Bigr\|^2$

This ensures balanced optimization between local rigidity and accurate vertex positioning. We will clarify this explicitly in the revised manuscript.

[Q3] $v_t$ and $v^{(k+1)}_{h}$ are fixed. What is the learnable in Eq (8)? Is it $\lambda$?

[A3] Yes, $v_t$ and $v^{(k+1)}_{h}$ are both fixed. Additionally, as explained in A1-2, the parameter $\lambda$ is a manually set hyperparameter (set as 0.2) and not learnable. Eq. (8) itself contains no learnable parameters. The learnable parameters are the positions of vertices (other than handle and target vertices), which are optimized through the SR-ARAP energy function with the handle-matching term in Eq. (9).

[Q4] The paper mentions using both DepthMesh and DiffMesh. Which one is preferred and why? Additionally, which mesh (DepthMesh or DiffMesh) is used to achieve the reported results in Tables 1, 2, and 3?

[A4] We provide additional experimental comparisons of drag-editing results using DepthMesh and DiffMesh in Fig. 12 (please refer to the link provided below). As shown in Fig. 12(a), DepthMesh struggles to generate geometry for regions unseen in the single input image, resulting in unnatural mesh deformation. In contrast, DiffMesh (Fig. 12(b)), generated via a diffusion model, effectively captures complete geometry, enabling more natural and coherent mesh deformation. Therefore, we prefer DiffMesh due to its better geometric consistency, crucial for accurate mesh deformation using our SR-ARAP algorithm. All reported results in Tables 1, 2, and 3 are based on DiffMesh. The detailed editing process using DiffMesh is illustrated in Fig. 11.

Fig 11. https://anonymous.4open.science/r/FlowDrag-B950/Fig_11.pdf

Fig 12. https://anonymous.4open.science/r/FlowDrag-B950/Fig_12.pdf

[Q5] Section C of the supplementary mentions background separation. How is it achieved?

[A5] Background separation is performed during the DepthMesh generation by applying a background threshold ($τ_b$) in Step 4 of Algorithm 1 (Supplementary, Section B). Specifically, any mesh facets with depth values smaller than $τ_b$ are removed. For example, Fig. 7(a) in Section C illustrates a result without background separation ($τ_b=0$), whereas Fig. 7(b), with $τ_b=0.3$, effectively removes background facets.

[Q6] What are the hardware requirement for FlowDrag?

[A6] FlowDrag requires less than 14GB of GPU memory for processing a 512×512 input image, as evaluated on a single NVIDIA A100 GPU. We will include detailed hardware requirements in the final manuscript.

[Q1] The method facilitates overall geometric consistency, but edited images still show some fine-grained inconsistencies, e.g., the hat shape in the first sample of Fig.6.

[A1] Yes, we agree with the reviewer’s observation. While FlowDrag significantly improves overall geometric consistency, some fine-grained geometric inconsistencies remain (e.g., the hat shape in Fig. 6). We speculate that this issue primarily arises due to the inherent limitation of the pre-trained 2D Stable Diffusion model (version 1.5), specifically its insufficient 3D understanding. This limitation is commonly observed across diffusion-based drag editing methods. Nonetheless, FlowDrag mitigates overall geometric inconsistency and shows enhanced robustness compared to existing methods (please refer to detailed comparisons in Tables 1 and 2, as well as Figures 6 and 9). To further illustrate this strength, we provide additional visualization results in Fig. 10 (additional comparisons on VFD-Bench and Drag100 datasets, including extra examples from Prompt-to-Prompt) and detailed visualizations of our mesh-guided editing process in Fig. 11. Additionally, we believe that utilizing backbones inherently capable of stronger geometric reasoning (e.g., video diffusion models) could potentially address and further reduce such fine-grained inconsistencies. We hope our work and results inspires future research in this promising direction.

Fig 10. https://anonymous.4open.science/r/FlowDrag-B950/Fig_10.pdf

Fig 11. https://anonymous.4open.science/r/FlowDrag-B950/Fig_11.pdf

[Q2] How important and robust is the 3D mesh reconstruction? Additional ablations or insights would help clarify this.

[A2] To analyze the importance and robustness of the 3D mesh reconstruction, we conducted additional sensitivity analyses for both DepthMesh and DiffMesh, as shown in Fig. 13 (please refer to the link provided below). For DepthMesh (Fig. 13(a)-(b)), robustness is evaluated by varying the reduction ratio, which directly controls the density of facet connections during mesh construction (as detailed in Supplementary Algorithm 1, Step 3). Specifically, a ratio of 1 indicates a fully connected mesh, while lower ratios substantially reduce vertices and facets, causing geometry degradation and unintended outcomes in mesh deformation and subsequent drag editing. We performed experiments on 20 images from DragBench. Since DragBench lacks ground-truth edited images, we quantified robustness by computing the ratios of metrics relative to the highest-quality mesh (reduction ratio = 1, used as reference image), defined explicitly as follows:

$\text{PSNR ratio} = \frac{\text{PSNR (source image)}}{\text{PSNR (reference image)}}$

$\text{1–LPIPS ratio} = \frac{\text{1–LPIPS (source image)}}{\text{1–LPIPS (reference image)}}$

As shown in Fig. 13(a)-(b), FlowDrag demonstrates stable and robust editing outcomes within an effective reduction ratio range (0.001–1). For DiffMesh (Fig. 13(c)-(d)), we similarly assessed robustness by varying the diffusion sampling steps (40, 20, 10, and 5) in the image-to-3D mesh generation process (using Hunyuan3D 2.0). Evaluations on the same 20 DragBench images with identical metrics revealed that sampling steps between 10 and 40 consistently maintained overall object geometry. However, at sampling steps below 10 (e.g., step=5), geometry degraded significantly, causing unintended deformation and editing results. Nevertheless, FlowDrag showed strong robustness for sampling steps of 10 or higher (Fig. 13(c)-(d)). These detailed analyses confirm FlowDrag’s robustness across various mesh reconstruction conditions, validating our method’s effectiveness.

Fig 13. https://anonymous.4open.science/r/FlowDrag-B950/Fig_13.pdf

[Q1] The authors do not discuss the impact of 3D mesh construction. Could failures or severe artifacts in mesh reconstruction cause image editing to fail?

[A1] Yes, severe artifacts or failures in 3D mesh reconstruction could hinder the accurate generation of an accurate 2D vector flow, potentially causing editing failures. To address this concern, we evaluated the robustness of FlowDrag across various mesh reconstruction conditions, as detailed in our response to Reviewer 1 (A2) and illustrated in Fig. 13 (please refer to the link provided below). Due to the space limitations, we respectfully ask the reviewer to refer to these detailed analyses. Briefly, we analyzed two mesh-generation approaches used in FlowDrag: DepthMesh and DiffMesh. For DepthMesh, varying the reduction ratio (controlling mesh density during construction) can degrade the original geometry if reduced excessively. For DiffMesh, changing the diffusion sampling step of the image-to-3D diffusion model (Hunyuan3D 2.0) can introduce artifacts and geometry degradation when sampling steps become very short. Our comprehensive experiments identified robust operating ranges: DepthMesh remains robust within a reduction ratio range of approximately 0.001–1, and DiffMesh maintains robustness for diffusion sampling steps of 10 or higher. These analyses demonstrate FlowDrag's robustness, confirming its capability to consistently produce stable and accurate editing outcomes, even under varying conditions of mesh reconstruction quality.

Fig 13. https://anonymous.4open.science/r/FlowDrag-B950/Fig_13.pdf

[Q2] Regarding the selection of control points, as seen in Figure 3, the geometry of the edited dog's head is inconsistent after mesh editing. If control points are chosen in a regular manner, could this lead to bad results?

[A2] As shown in Fig 3, minor geometric inconsistencies may arise from the mesh deformation itself, since the deformed mesh does not perfectly preserve fine-grained rigidity. However, we do not directly use this mesh for editing. Instead, we project the deformed mesh onto a 2D vector field and select optimal control points (referred to as "2D vector flow") from this projection. Furthermore, we conducted the experiment on selecting control points in a regular manner corresponding to the "Uniform sub-sampling" approach described in our paper (Sec 4.3). Our experiments show that this approach still effectively maintains overall geometric consistency and outperforms many existing methods. (We guess this effectiveness is attributed to our multiple drag vector concept.) Additionally, we proposed "Magnitude-based sampling", which selects more effective vectors with the largest displacements, achieves optimal editing results, as quantitatively demonstrated in Table 5.

[Q3] The paper presents limited qualitative visualization results.

[A3] We provide additional qualitative visualization results in Fig. 10 (additional results on VFD-Bench and Drag100), Fig. 11 (visualization of the mesh-guided editing process using DiffMesh), Fig. 12 (comparison of mesh deformation pipelines using DepthMesh and DiffMesh), Fig. 13 (sensitivity analysis and robustness comparison of mesh deformation), and Fig. 14 (additional qualitative comparisons of drag-based editing results). We will also incorporate these results in the final manuscript.

Fig 10. https://anonymous.4open.science/r/FlowDrag-B950/Fig_10.pdf

Fig 11. https://anonymous.4open.science/r/FlowDrag-B950/Fig_11.pdf

Fig 12. https://anonymous.4open.science/r/FlowDrag-B950/Fig_12.pdf

Fig 13. https://anonymous.4open.science/r/FlowDrag-B950/Fig_13.pdf

Fig 14. https://anonymous.4open.science/r/FlowDrag-B950/Fig_14.pdf