Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.

W1 & Q1. While the proposed method shows promising performance, an analysis of computational efficiency (e.g., number of parameters, FLOPs) is missing. A comparison of resource usage with baseline methods is necessary to demonstrate that the framework achieves its results without excessive computational overhead

Firstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.

Also, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the length of frames.

Settings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16

W2 & Q3. Ablation analysis is insufficient. It is important to provide a detailed breakdown of how each module contributes to the overall performance improvement, based on the ablation results. As discussed in Section 4.3, the authors should provide concrete examples and analysis of specific scenarios where this augmentation brings actual benefits, along with visual or quantitative evidence to support MA’s effectiveness.

Thank you for your valuable feedback. We sincerely apologize for the lack of clarity in our explanation of Tab.1. To clarify, the results presented in the table are cumulative, meaning that each module, i.e. MRG, MA, and DMP, was added sequentially. Therefore, the specific contribution of the DMP module can be observed by comparing the last two columns of the table.

Firstly, to further demonstrate the standalone accuracy and effectiveness of the DMP module, we conducted additional experiments, as shown in the first table below. The baseline setting employs the "mask-and-inpaint" paradigm without incorporating any of the proposed modules or strategies.

From the experimental results, we can clearly observe the quantitative improvements achieved by introducing the difference mask predictor (DMP) module to supervise the training process. This further validates the effectiveness of our proposed approach.

Secondly, to validate the impact of our synthetic dataset and assess the role of the backbone model, we conducted an ablation study on DiffuEraser. Specifically, we compared its performance before and after training with our dataset. The results (shown in the second table below) indicate that while our dataset contributes to measurable improvements in DiffuEraser’s performance, the overall enhancement remains limited. This observation motivated us to explore a more robust backbone model to achieve higher performance.

Thirdly, to support MA’s effectiveness, we conduct an experiment in inference, comparing the effect of output video while dealing with ground-truth masks and imperfect masks generated by SAM2. The quantitative results can be seen in the subsequent third table. And from the table we can find that by adopting MA in training process, our model performs robustness while the input masks are not perfect. For visual results proving the robustness, we’re sorry that we can’t provide them currently for the requirements of conference committee and we promise we’ll supplement them later in our open-source project page.

Table 1: Ablation studies on each module

Table 2: Ablation studies on the backbone model

Table 3: Experiments on the input masks during inference

Q2. The motivation in lines 156–159 clearly justifies the necessity of MRG, and its effectiveness is well demonstrated through the "+MRG" results in Tab. 1. Given this, it would be interesting to see whether similarly dramatic improvements can be observed when mask region guidance is applied to other methods as well. Do baseline models like ProPainter or DiffuEraser exhibit the same performance gains when MRG is incorporated?

Thank you for your insightful observation. Indeed, we initially explored the possibility of using the models you mentioned. However, we encountered several challenges that led us to reconsider their suitability for our framework.

First, ProPainter relies on an image propagation approach, which requires a masked video as input. This design is fundamentally incompatible with our mask region guidance paradigm, making further experimentation with ProPainter impractical for our specific use case.

As for DiffuEraser, its dual-branch architecture depends on a pretrained image inpainting model (BrushNet). Specifically, its inpaintng ability relies on the pretrained BrushNet's image inpainting ability and to adapt MRG for training, it needs to change BrushNet's architecture completely and involve more additional design, which makes it difficult to conduct a reasonable comparison experiment. Given these constraints, we decided against pursuing further experiments with these two models.

Q4. I’m curious whether masking only part of an object results in side effects being removed only for that region, or if the effect of mask augmentation is strong enough to eliminate all related side effects. For example, in a scene where a person and a horse are jumping over a fence, what would the result look like if only the person is masked out?

Thank you for raising this interesting question. The effect of partial masking depends on the specific context of the image and the relationships between objects.

For instance, in the scenario you described (Fig.7, line 2) where a person and horse are jointly jumping over a fence, masking only the person would successfully remove just the human figure while preserving the horse. This demonstrates that when objects are distinctly separate entities, partial masking can produce localized effects.

However, when objects are intrinsically connected, the outcome differs. As shown in Fig.1 of the supplementary material featuring a cow with a neck knot, masking just the cow's body while leaving the knot unmasked produces the same result as masking both elements. This occurs because the knot is perceived as an integral part of the cow's anatomy by the model.

Hello, a gentle reminder for discussions. Authors have provided a rebuttal, and it would be great if you could acknowledge it and comment whether it addresses your concerns!

Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.

W1. Fig. 1 is not clear when I zoom in, please use PDF images.

We apologize for the quality of figure and will replace it with clear PDF file in the revised version.

W2. The synthetic dataset only includes videos with camera motion and without object motion, which limits the capability of processing videos with moving objects. This point is only mentioned in Sec. 6, no visual results are shown in the paper.

Firstly, though there lack videos with moving object in the synthetic dataset, ROSE still show great performance on erasing moving object, as shown in Fig.1 in the supplementary material. The performance could be attributed as follows: The main purpose of ROSE is to guide the video model to distinguish the side effect area of selected object and conduct joint erasing. Erasing the object itself is not the main challenge of ROSE, which requires to fill the object area with proper video dynamics. Therefore, ROSE shows generalized performance on either moving or static objects since the object masks are provided, focusing on erasing the side effect areas. Furthermore, videos with camera motion help to construct relative object motion, which also enhance the model's performance on moving objects.

Secondly, we show the limitations of ROSE in Fig.1 (line1-8) of supplementary material, showcasing objects with large motion or non-rigid motions. Though sometimes showing filckering effects, the overall performance of ROSE is still reasonable in many scenarios with substantial motions.

Thirdly, for future improvement, since constructing synthetic video with object motion is challenging, it could be explored to combining pseudo real-world data or generated data into the dataset for further improve the generalization of ROSE.

W3. The side effects are divided into six types, but only three types are shown in Fig. 7.

Thanks for the suggestion. In Fig.7 of the main paper, we show four types of side effects (shadow, common artifacts, reflection, and light source). In Fig.2 of the supplementary material, we present further visual comparisons covering reflection, shadow, light source, and mirror effects. We acknowledge that the translucent category was not included. We have prepared the visualization results on this category. Since no images can be attached in the rebuttal, we will add them in the revised version.

W4. The lack of experiments on investigating the effectiveness of the mask-based filtering.

Thank you for your observation regarding the mask-based filtering method presented in Fig.2. Below, we provide a detailed explanation of its purpose and implementation:

Objective of Mask-Based Filtering

In our synthetic dataset construction pipeline using Unreal Engine, efficiency is critical due to the large-scale generation of video pairs. A key challenge is ensuring the quality of these pairs by identifying and filtering out problematic cases, such as those with severe occlusion.

Implementation Details

（1）We first render videos alongside their corresponding object masks.

（2）By calculating the ratio of masked pixels to total pixels in each frame, we quantitatively assess occlusion levels.

（3）Camera trajectories resulting in occlusion ratios beyond a predefined threshold are automatically discarded.

Deterministic Nature of the Process

Since the filtering mechanism is entirely rule-based and deterministic, the outcome is fully predictable given the input masks and thresholds to ensure purpose that the targeting object exists in the video.

This approach significantly accelerates dataset construction by eliminating manual inspection, while ensuring consistent quality across all generated video pairs. We appreciate your attention to this methodological detail and hope this clarification addresses your question.

W5. Sec. 4.3 claims that the user-provided masks are often imperfect and thus proposes mask augmentation to mitigate this issue. But there is no such case in the experiment to show how ROSE could deal with the imperfect masks.

Thank you for your valuable feedback. We appreciate your observation regarding the handling of imperfect masks in our experiments. Due to the conference committee's requirements, we were unable to include specific examples demonstrating ROSE's performance with imperfect masks in the current submission. However, we plan to supplement these cases in future work to provide a more comprehensive evaluation.

Real-World Application

In practice, our framework leverages SAM2, a state-of-the-art segmentation model, to generate high-quality masks based on user input. SAM2's robustness ensures that most masks are accurate, but even for imperfect cases, ROSE is designed to:

（1）Automatically infer the complete object mask from partial or noisy inputs.

（2）Produce plausible and stable results despite mask inaccuracies.

Experimental Validation

To quantitatively assess ROSE's resilience to imperfect masks, we conducted an additional experiment comparing results using ground-truth masks and results using intentionally imperfect masks generated by SAM2. The quantitative results (shown in the table below) confirm that ROSE maintains strong performance even with imperfect mask inputs, underscoring its practical applicability.

We hope this clarification addresses your concern, and we welcome further discussion on this point.

Table 1: Experiments on the input masks during inference

W6 & Q2. The visual comparisons are superior in Fig. 7, but the PSNR value in Tab. 2 is lower than ProPainter. Does the low PSNR in Tab. 2 mean that ROSE cannot generalize well to the real data due to the synthetic training dataset?

The evaluation data of Tab.2 is constructed using "copy-and-paste" approach to create real-world video pairs. The results only reflect the performance of erasing object itself. ROSE shows comparable results on PSNR and better results on SSIM and LPIPS, which demonstrates that ROSE can effectively remove the object in real videos.

However, the results of Tab.2 does not reflect the side effect erasing, which is the core objective of ROSE. Through extensive visual comparisons and quantitative experiments in Fig.2 and Tab.3, we have demonstrated that our model excels in this key capability.

Also, while ProPainter achieves a marginally higher PSNR score, our model demonstrates superior overall performance by outperforming it in both SSIM and LPIPS metrics. This balanced excellence across all three standard evaluation criteria confirms that our method delivers more structurally accurate results, produces visually superior outputs and maintains competitive fidelity.

To sum up, with comprehensive comparison on the three subsets evaluation of ROSE-Bench, we demonstrate that ROSE achieves satisfying performance on both side effect and object removal on real-world and synthetic videos.

Q1. Does ROSE-Bench include similar imperfect masks as well to better support the function of mask augmentation?

Thank you for your question regarding our mask augmentation approach.

We sincerely apologize that we are unable to provide visual demonstrations of the mask augmentation effects at this time due to the requirements of the conference committee.

To substantiate the effectiveness of our mask augmentation method, we have included comprehensive quantitative results in the table under W5's answer. The data clearly demonstrates our model's robust performance when processing imperfect masks. The empirical evidence confirms that our augmentation strategy successfully enhances the model's capability to handle various mask imperfections while maintaining stable output quality.

Hello, a gentle reminder for discussions. Authors have provided a rebuttal, and it would be great if you could acknowledge it and comment whether it addresses your concerns!

Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.

W1. While the paper presents strong quantitative and qualitative comparisons, it lacks video-based results, which are crucial for evaluating temporal consistency and perceptual quality in video inpainting. The current comparisons rely mostly on metrics and static frame visualizations, which may not fully reflect performance on dynamic content. Including sample videos would strengthen the evaluation.

Thanks for your suggestions. We mainly show temporal sequential results in Fig.1 of main paper and Fig.1&2 in supplementary material, which could briefly reflect the erasing performance on dynamic content. These frame sequences demonstrate temporal consistency of ROSE's outputs and high-qualified video reconstruction. Due to conference committee requirements, we are unable to share full video demonstrations now, and will present more results in both revised version and open-sourced project page. We appreciate your understanding and welcome further discussion on these results.

W2. Additionally, the paper does not report inference time or model size, making it difficult to assess the method’s efficiency and scalability compared to prior work.

Firstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.

Secondly, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the frames.

Settings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16

W3. To better validate the effectiveness of the constructed dataset, the authors should consider training baseline video inpainting models (e.g., DiffuEraser or ProPainter) on the same synthetic data and comparing their improvements. Since ROSE builds on top of Wan, a powerful generative model, it’s unclear how much of the performance gain comes from the dataset versus the backbone.

Thank you for your valuable suggestion. To validate the effectiveness of our synthetic dataset, we conducted preliminary experiments by training DiffuEraser on the same data and comparing its performance with the original model. The results demonstrated that the trained DiffuEraser exhibited some capability in mitigating object removal side effects.

However, due to the conference committee's requirements, we are currently unable to include the visual comparison results in this submission. We plan to supplement these details in the final version of the paper.

During our experiments, we observed that the performance of the trained DiffuEraser, while improved, did not fully meet our expectations. Since our focus is more on the removal effect of side effects, DiffuEraser after training didn't give satisfactory visual results. Also, we provide a table below which quantitatively shows the enhancement of DiffuEraser after training on the synthetic dataset. This led us to explore alternative base models, and we ultimately adopted Wan due to its stronger generative capabilities. The switch to Wan enhanced the model's overall performance, confirming that the choice of backbone architecture plays a critical role in the task. Also we shows the improvement made by changing the backbone model to Wan. In this table, we trained Wan with baseline settings, which uses "mask-and-inpaint" paradigm and without using any strategies and modules we design.

To further boost the model's robustness and inpainting quality, we introduced additional techniques, including mask region guidance, mask augmentation, and a difference mask predictor. These innovations collectively address the limitations observed in earlier experiments and contribute to the superior results presented in our work.

Table 1: Ablation studies on the backbone model testing on synthetic paired data

Hello, a gentle reminder for discussions. Authors have provided a rebuttal, and it would be great if you could acknowledge it and comment whether it addresses your concerns!

Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.

W1 & Q2. Limited Real-World Validation: The model’s performance on truly in-the-wild videos is insufficiently analyzed. Quantitative comparisons for real-world unpaired settings are lacking, and failure modes, particularly flickering under motion, are noted but not deeply explored.

In Tab.4, we have evaluated real-world unpaired data with quantitative metrics from VBench. Since the real-world unpaired dataset lacks the ground-truth edited video, we can’t evaluate with metrics like PSNR, SSIM and LPIPS. To comprehensively evaluate ROSE’s performance on real-world scenarios, we constructed real-world paired dataset as mentioned in Sec.5.1 which has the ground-truth edited video we needed for evaluation. And we apologize for that currently we can't provide some visual examples of failure where the videos exist flickering under very large motion for the requirements of conference committee. Later we'll supplement more examples including failure cases in the paper and our open-source project page.

W2 & Q4. Computational Demands: Both training and inference are resource-intensive due to the diffusion-based backbone and large-scale rendering pipeline. Practical deployment concerns are not addressed. What are the model’s runtime characteristics (e.g., FPS, memory usage) on long videos? Are there strategies for reducing compute requirements?

Firstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.

Secondly, the rendering pipeline is specifically designed for dataset preparation and does not contribute to runtime computational costs. Rendering a 6-second video (15 fps, 1920×1080 resolution) takes approximately 9 seconds.

Thirdly, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the frames.

Settings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16

W3 & Q3. Ablation Gaps: While key components are evaluated, there is limited analysis of the difference mask predictor’s standalone effectiveness or its per-category contributions. What is the standalone accuracy or effectiveness of the difference mask predictor across different types of side effects? The backbone model’s role is also not isolated.

Thank you for your valuable feedback. We sincerely apologize for the lack of clarity in our explanation of Tab.1. To clarify, the results presented in the table are cumulative, meaning that each module, i.e. MRG, MA, and DMP, was added sequentially. Therefore, the specific contribution of the DMP module can be observed by comparing the last two columns of the table.

Firstly, to further demonstrate the standalone accuracy and effectiveness of the DMP module, we conducted additional experiments, as shown in the first table below. The baseline setting employs the "mask-and-inpaint" paradigm without incorporating any of the proposed modules or strategies.

From the experimental results, we can clearly observe the quantitative improvements achieved by introducing the difference mask predictor (DMP) module to supervise the training process. This further validates the effectiveness of our proposed approach.

Secondly, to validate the impact of our synthetic dataset and assess the role of the backbone model, we conducted an ablation study on DiffuEraser. Specifically, we compared its performance before and after training with our dataset. The results (shown in the second table below) indicate that while our dataset contributes to measurable improvements in DiffuEraser’s performance, the overall enhancement remains limited. This observation motivated us to explore a more robust backbone model to achieve higher performance.

Table 1: Ablation studies on each module

Table 2: Ablation studies on the backbone model

W4. Unclear Sim-to-Real Transfer: The domain gap between synthetic training data and complex real-world scenarios is acknowledged but not quantitatively assessed. The generalization capacity of ROSE remains uncertain.

We acknowledge the potential domain gap between synthetic training data and complex real-world scenarios. However, our model demonstrates strong generalization capabilities in the diverse real-world scenarios we tested.

To rigorously assess our model’s real-world performance, we conducted quantitative evaluations using both paired and unpaired real-world data:

• Paired Data (Tab. 2): Using a "paste-and-copy" approach to construct paired real-world data, we measured the model’s ability to remove objects and inpaint the affected regions. The metrics confirm high performance in these tasks.
• Unpaired Data (Tab. 4): For scenarios lacking ground-truth videos, we employed VBench metrics (e.g., video quality, temporal consistency) to evaluate the model’s output. These results further validate our model’s robustness and generalization capacity in real-world domains.

While the domain gap remains a challenge in sim-to-real transfer, our extensive visual and quantitative results demonstrate that ROSE generalizes effectively to the tested real-world scenarios. We will continue to explore broader conditions to further bridge this gap.

Q1. Domain Gap: How does ROSE’s performance degrade when applied to uncurated, real-world videos with side effects outside the defined categories? Have you considered domain adaptation techniques to bridge this gap?

Thank you for your insightful question regarding the domain gap and ROSE's performance on uncurated real-world videos.

First, as demonstrated in Fig. 1 and Fig. 7, ROSE exhibits strong performance across both synthetic and real-world domains, outperforming baseline models in most scenarios. During testing, we observed that the model adapts well to real-world videos, though it encounters challenges with highly complex side effects such as ripples (Fig. 1, rows 2–4) where the results, while improved, may not be flawless.

While we have not yet implemented advanced domain adaptation techniques, we are open to suggestions. Your recommendations on specific methods to bridge this gap would be highly valuable.

We appreciate your feedback and would welcome any further insights you might have.

Q5. Generalizability: How transferable is ROSE to new video domains, such as mobile or drone footage, or to settings with unseen environmental side effects?

Thanks for your suggestion. First, we evaluate ROSE across more real-world video domains, including mobile, drone footage, and underwater scenes. The results indicate robust performance of ROSE in these scenarios. Due to conference committee requirements, we are currently unable to share visualization results, and will present them in the revised version.

Second, while the model handles known side effects effectively, its performance on unseen types remains an area for further exploration. For example, when dealing with the scenarios where objects have more complex physical interactions with the environment like ripples generated by a duck and water vapor generated by hot food, our model may fail to perceive those intricate interactions and thus can't remove those side effects together with objects. But with current experience, we will include such limitations in revised version, and will leverage a wider range of synthetic technologies to address these issues in future works.

Hello, a gentle reminder for discussions. Authors have provided a rebuttal, and it would be great if you could acknowledge it and comment whether it addresses your concerns!