Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editing

We appreciate Reviewer-ZGWC for the insightful comments. We address the main concerns bellow.

Better baseline (DreamVideo [1] and MotionBooth [2] )

Thanks. Our task aims to enable one shot video editing based on concept text and video while the mentioned works DreamVideo [1] and MotionBooth [2] attempt to solve image to video generation, which clearly differs from our task. So it is inappropriate to compare DreamVideo [1] and MotionBooth [2] as the baseline.

Zeroscope is open-sourced video diffusion model instead of video editing method

Thanks for pointing this out and we will fix it in the revision.

Why image diffusion (stable diffusion) is used as a backbone

Good question ! In fact, we did try the experiment to apply concept augmented textual inversion to a video diffusion model, such as Zeroscope, but the results are not good as we expected. We guess that the generalization of concept word may easily be coupled with the frame length of concept video in the inversion process. In other words, the inversion results contain temporal information of the concept video, which is obstructive to be applied to edits for other videos. Actually, spatial information of the concept video is what we need, and we can randomly sample frames in the concept video and input them to a image diffusion backbone for the inversion process to avoid or break this coupling, which refers to Lines 288-290. Besides, using an image diffusion model as a backbone to extend with temporal modules can save VRAM overhead either for CATI or DPS on one Nvidia RTX4090 device.

L242-245, can the authors show results?

Thanks. The dispersion phenomenon of cross-attention between concept word and video latents in Fig.5, in which (a), (b) and (c) are more likely to appear the dispersion phenomenon due to lack supervision. The full results are shown in Fig.11, Fig.12 in Appendix.3.

CLIP consistency fails to reflect temporal consistency. Using other metrics like DINOv2 (like the ones from VBench[3]) or LIPIPS would be better

Good suggestion. We conduct additional quantitative evaluation using DINOv2 and LPIPS metrics reveals that MotionDirector indeed achieves better temporal consistency. However, MotionDirector's highly stochastic nature leads to less stable video editing results, particularly in maintaining non-target area consistency and source-target object matching fidelity. Our method demonstrates superior performance in these crucial aspects, showing an average improvement of 4.06 M-PSNR over all baseline methods. The results are shown below.

For editing w/ concept video

For editing w/o concept video

The concept augmented textual inversion (CATI) lacks technical contribution

Thanks. We would like to clarify that we aim to flexibly edit source videos using concept videos as a key task setting, with our primary motivation being to enrich the latent space through one-shot concept videos to achieve more faithful inversion representations. We validate this improvement through ablation studies, as shown in Fig. 4. While LoRA and Textual Inversion have been widely used separately in image style transfer tasks, to the best of our knowledge, no existing variant of textual inversion simultaneously employs LoRA tailored for video editing task. Existing approaches that do fine-tune the denoising network face significant challenges in one-shot concept video scenarios, such as overfitting and high memory requirements during training. Our approach addresses these limitations by leveraging LoRA modules to fine-tune attention value weights with a smaller learning rate, maintaining low VRAM overhead during tuning while preserving plug-and-play capabilities. We believe CATI is technical novel for these reasons.

Dear Reviewer ZGWC,

Hoping this message finds you well. Thank you once again for your thoughtful and thorough feedback on our work. We greatly appreciate the time you have invested in reviewing our manuscript. In response to your comments, we have made substantial revisions and added additional experiments to better address the concerns you raised.

As the rebuttal-discussion period draws to a close, we kindly ask if you could review our response, especially the details in this document link, to confirm that we have adequately addressed your concerns. Your feedback on our response will be invaluable to us in ensuring that we have resolved all outstanding issues.

If there are any further questions or points you would like us to clarify, please feel free to let us know. We are committed to refining our work based on your input, and your timely feedback would be greatly appreciated.

We sincerely hope that you will consider the contributions our research brings to the field of one-shot video editing. Thank you again for your insightful comments and for your continued support.

Best regards,

The Authors

The reviewer thank the authors for the answers. The reviewer has carefully read the paper, including the authors’ responses and subsequent discussions, and assures the authors that all answers are being thoughtfully analyzed. There is no need to reiterate requests for reviewers to provide responsible scores based on the responses, as this is already a standard practice.

1. Clarification on Baselines: The reviewer appreciates the authors' openness to addressing practical concerns. If the authors believe that a comparison with DreamVideo is of little practical significance, this is acceptable. However, the reviewer felt it was necessary to correct the mischaracterization that "DreamVideo [1] and MotionBooth [2] aim to solve image-to-video generation." As previously mentioned, the primary concern is the choice of stronger baselines. Before the release of foundational video diffusion models like ZeroScope or VideoCrafter, it was common to adapt Stable Diffusion for video editing tasks. Since mid-2023, however, the field has seen a proliferation of baselines leveraging text-to-video backbones. Are the authors confident that TAV, FateZero, and RAVE represent state-of-the-art (SOTA) or near-SOTA baselines for comparison at this point? Providing a more robust comparison with stronger baselines would significantly strengthen the paper, rather than focusing on justifying the choice of weaker baselines that lack temporal consistency (like the authors' method).

2. Handling Spatio-Temporal Learning: The reviewer is unsurprised that naive attempts to extend the ZeroScope backbone have not yielded promising results. The issue of entangled spatio-temporal learning when applying the full video diffusion loss—regardless of whether embeddings or weights are being optimized—is well-documented in the literature. The critical factor lies in the optimization approach: A commonly accepted approach to decouple spatial and temporal dimensions is to disable the spatial layers of the video diffusion model and train by sampling single frames from the reference video [1,2,3,4]. The reviewer believes this established method could effectively address the spatio-temporal coupling issue. Therefore, the authors’ explanation for the observed failure modes does not seem entirely convincing.

3. Co-Training Text Embeddings: For the same reasons outlined in point (2), to my unprofessional and humble opinion as claimed by authors, the reviewer questions the necessity of co-training text embeddings with LoRA for one-shot video editing tasks. Numerous works in the field, including those targeting appearance or motion customization (or both), have demonstrated success without optimizing text embeddings. Examples include works [1,2,3,4,5,6], which have achieved effective video customization by focusing solely on appearance or motion learning. The reviewer therefore finds the authors’ rationale for co-training text embeddings unconvincing, particularly given the availability of well-established alternative approaches.

[1] MotionDirector: Motion Customization of Text-to-Video Diffusion Models

[2] Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models

[3] MotionCrafter: One-Shot Motion Customization of Diffusion Models

[4] ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning

[5] VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models

[6] Customizing Motion in Text-to-Video Diffusion Models

Thank you for the reviewer’s feedback. We would like to summarize our understanding of the points raised in the response (please feel free to point out any inaccuracies):

(1) Reviewer acknowledges that of DreamVideo and MotionBooth initially suggested for baseline comparison are not suitable when compared to our method.

(2) Reviewer recognizes the validity of our explanation regarding ZeroScope, but suggests that our reasoning could be further strengthened, particularly with reference to the common practice of disabling the spatial layer of video diffusion models.

(3) The reviewer continues to question the necessity of using both concept text and concept video as inputs based on existing works.

We will address remaining points in the following responses.

(1) In response to the reviewer’s question, "Are the authors confident that TAV, FateZero, and RAVE represent state-of-the-art (SOTA) or near-SOTA baselines for comparison at this point?"

— Yes, we are confident in our selection of baselines. We humbly invite the reviewer to provide specific citations or examples of alternative approaches that could challenge the methods we have used. This will help us further refine our work and ensure our comparisons remain rigorous and comprehensive.

(2) We sincerely appreciate the reviewer’s suggestion regarding the "commonly accepted approach." We tested this in our setting, but it did not produce satisfactory results. Nevertheless, we recognize the value of including such comparisons for completeness and will incorporate them in the final version of our work.

(3) While the existing text2video approaches represents an established paradigm, it is not necessarily incompatible with our methodology. This is because solely using text guidance, though functional to some extent, lacks control and flexibility. For instance, if we aim to embed a specific object from a concept video into a source video, purely text-based guidance falls short in achieving this goal. This is precisely why we incorporate concept pairs in our method—to make the editing process more controllable and adaptable to nuanced editing scenarios.

We thank reviewer G4M5 for the feedback. We appreciate reviewer G4M5 for "comprehensive experiments" and "clear writing". We address the main concerns bellow.

Limited innovation

Thanks. (I) We would like to clarify that we aim to flexibly edit source videos using concept videos as a key task setting, with our primary motivation being to enrich the latent space through one-shot concept videos to achieve more faithful inversion representations. We validate this improvement through ablation studies, as shown in Fig. 4. While LoRA and Textual Inversion have been widely used separately in image style transfer tasks, to the best of our knowledge, no existing variant of textual inversion simultaneously employs LoRA tailored for video editing task. Existing approaches that do fine-tune the denoising network face significant challenges in one-shot concept video scenarios, such as overfitting and high memory requirements during training. Our approach addresses these limitations by leveraging LoRA modules to fine-tune attention value weights with a smaller learning rate, maintaining low VRAM overhead during tuning while preserving plug-and-play capabilities. We believe our proposed approach is technically novel and as also appreciated by Reviewer-HLfd "The motivation is clear". (II) Our dual prior supervision serves as a supervision mechanism designed to reduce inconsistencies in non-target areas before and after video editing, addressing the low consistency in existing approaches, especially in editing source video with concept video (As illustrated in Tab.1 in our draft. The reviewer ZGWC also mentioned that "the SCAM and TCAM is novel" ).

The concern about the quality of generated videos

Thanks. (I) Regarding video quality, we would like to highlight that in the challenging task of video editing with concept videos, our method demonstrates significant advantages in maintaining consistency of non-target areas - a critical aspect where existing methods often struggle. Specifically, our approach achieves an average improvement of 4.06 M-PSNR over baseline methods in preserving non-target regions, while ensuring faithful concept transfer. (II) For the video length limitation, this is primarily due to hardware constraints rather than methodological limitations. Our current experiments are conducted under limited computational resources(only one RTX 4090 with 24G memory), which necessitate certain practical compromises: using a moderate frame sampling step and shorter sequences to manage memory usage, working with standard rather than high-resolution videos. (III) we provide additional results including denser frame sampling in https://anonymous.4open.science/w/STIVE-PAGE-B4D4/ with more powerful device (Nvidia L20 with 48G memory), the results demonstrate that our approach can be readily extended to process longer videos at higher resolutions with denser frame sampling, as the method itself is scalable.

Missing details on the training set

Thanks. Our method is devised for one-shot video editing. That is to say, the training set is only one source video to be edited and one concept video coupled with text template under the setting with concept video, while the training set is only one source video and text template under the setting of without concept video.

Unconventional and somewhat outdated metrics

Thanks. We kindly disagree with the comment. The Masked Peak-Signal-Noise Ratio (M-PSNR) is a crucial metric that intuitively and effectively reflects the consistency of non-target regions before and after video editing with Reviewer-ZGWC appreciate it as "convincing metric". Additionally, Concept Consistency leverages a multimodal CLIP visual encoder to encode objects from both the concept video and edited video results, measuring their feature similarity in vector space, which effectively reflects the faithfulness of concept video object transfer. We would appreciate detailed suggestions or citations for rational metrics that you believe might better evaluate these aspects.

About more ablation study results

Thanks. But we argue that some changes in attributes are indeed impressive in terms of visual quality and effect. This precisely demonstrates the most intuitive manifestation of our method's effectiveness in one-shot scenarios. Of course, it is not limited to color; other attributes, such as shape, are also included. We have provided additional results on More Ablation Section in the anonymous link https://anonymous.4open.science/w/STIVE-PAGE-B4D4/.

Dear Reviewer G4M5,

I hope this message finds you well. Thank you for your valuable feedback and thoughtful insights on our work. Your comments have helped us refine our approach, and we have incorporated additional analysis and experiments in the final version of the paper based on your suggestions.

As the rebuttal-discussion period is coming to a close, we would greatly appreciate it if you could kindly review our response (especially in https://anonymous.4open.science/w/STIVE-PAGE-B4D4/) to see if it adequately addresses your concerns. Your timely feedback would be immensely helpful in ensuring that we have properly clarified all points. If you have any further questions or suggestions, please do not hesitate to let us know.

Your decision is crucial to us, and we sincerely hope that you will recognize the contributions of this research to the field of one-shot video editing. We are very grateful for your time and consideration.

Best regards,

The Authors

We greatly appreciate that our "motivation is clear", and "the results are impressive", and "the paper reads well and is easy to follow". We address your concerns bellow.

Fair comparison with existing approaches. Good question ! (I) We obtain the results of existing approaches using the released open source code. (II) The methods we compared (like Tune-A-Video, FateZero and RAVE) are all based on the same CLIP text encoder that enables us to integrate the same input concept pairs into the models. For MotionDirector, it supports a spatial path to bring the object of concept video into latent space originally. That is to say, we can make fair comparisons for existing approaches. (III) Besides, we release the code of merging representation of object in concept video into the CLIP text encoder in our anonymous repository https://anonymous.4open.science/w/STIVE-PAGE-B4D4/

separate quantitative comparison table with and without concept video Thanks for the suggestion. We seperate the quantitative comparison into two part for using and not using concept videos to make it more clear as shown bellow. We will include this in the revision latter.

For editing w/ concept video

For editing w/o concept video

Missing references and some for comparisons

Thanks. These methods each have unique characteristics, primarily focusing on innovations in pretrained models, diffusion models, attention mechanisms, spatiotemporal consistency, and inter-frame relationships in video editing. For example, [1] AnyV2V uses a pretrained model to handle various video editing tasks, while [2] GenVideo employs a target-image-aware approach and novel InvEdit masks to overcome text-prompt limitations. Besides, [3] Factorized Diffusion Distillation introduces the EVE model by distilling pretrained diffusion models. For temporal consistency, [4] VidToMe merges self-attention tokens across frames. For spatial editing, [5] Pix2Video, [6] Slicedit, and [8] Cideo-P2P both improve results with attention features and diffusion models respectively. [7] TokenFlow leverages inter-frame correspondences to propagate features. Compared to existing approaches, our method focuses on attention supervision and control mechanisms and operates on a one-shot video editing paradigm. It also improves temporal consistency through extended temporal module parameters and enables the flexible integration of external concept objects. This approach not only differs from existing methods in task handling strategies but also complements them in principles and implementation details, thereby broadening the possibilities for video editing. We will include these references and make comparisons in the revision.

Detailed explanation for the pipeline of editing without concept video

Thanks. A pretrained lora/concept model is not required for editing without concept video. For example, we only use the word embedding in CLIP model to edit the "jeep" for "Porsche", then fine-tuning the denosing net with SCAM loss for source video and source prompt, and TCAM loss for source video and target prompt.

The range of concepts in CATI for generalization

Good question ! Our experiment shows that CATI can be applied to various objects, like robots and animals. In fact, it's not that difficult for objects in concept image or video to generalize well to new image or video. But for stable video editing (high consistency of non-target area and high matching faithfullness between source and target object), it is more suitable to edit rigid objects, like vehicles or other objects with less inner component movement.

We thank Reviewer yeHN for the constructive reviews. We address your concerns bellow.

overall quality of the edited videos is not satisfactory.

Good question. We acknowledge that the edited video does indeed contain some of the aspects you have carefully observed. But we guess the reason behind this lies in that we set a larger frame sampling step due to limited computational resources (only one RTX 4090 with 24G gpu memory) instead of only lacking video prior knowledge. Because, (I) we have actually validated video diffusion model at the very beginning. However, while the results demonstrate marginal advantages in the temporal, they fell far short of our expectation for scene editing (especially for spatial control). Therefore, we opt to use the standard Stable Diffusion coupled with temporal layers after careful consideration. (II) We conduct additional experiments with a small frame step with more powerfull devices (Nvidia L20 with 48G memory) in scenarios with weak temporal consistency, the edited videos are more smoother than before as shown in More Results Section of https://anonymous.4open.science/w/STIVE-PAGE-B4D4/. While our original experiments used a frame sampling step of 8 and sequence length of 6 frames, we test our approach with a streamlined step of 3 frames (i) and an extended sequence length of 14 frames (ii). The comparative analysis reveals the following:

for editing w/ concept video

for editing w/o concept video

(III) Although Flatten [1] guides video motion editing by introducing optical flow conditions, during inference, Flatten retains prior information such as convolutional and attention features from certain diffusion steps, which are then replaced during the denoising process to guide video editing. However, this process conflicts with operations such as attention swapping, making Flatten unsuitable for this method. TokenFlow [2] involves propagating estimated motion tokens to modify hidden states after extended attention computation, is fundamentally incompatible with our approach. Specifically, this propagation mechanism conflicts with our attention control strategy during the inference phase, making TokenFlow framework unsuitable for integration into our method.

The methodology is not strong for (I) concept-augmented textual inversion belongs to this variation by additionally finetuning V; (II) the scam and tcam loss although improves the results, does not seem novel or strong enough to boost the consistency of the videos.

Thanks for the very detailed comments. (I) We acknowledge that our approach is quite straightforward. However, it is not merely about replacing model parameters with V. More importantly, our analysis reveal that fine-tuning the denoising network in one-shot video editing scenarios is highly prone to overfitting and incurs significant computational costs. In contrast, fine-tuning V effectively addresses these issues, offering a simple yet efficient solution tailored to this specific task. We believe this is novel. (II) We propose the scam and tcam loss to improve the consistency in non-target areas before and after video editing, especially in editing source video with concept video, which is illustrated in Tab.1 and Fig.3 in the draft. This is also appreciated by the reviewer ZGWC "SCAM and TCAM is novel".

The efficiency is not optimal enough.

Good question ! But we would like to clarify that (I) It seems unfair to challenge efficiency because for the same setting : Tune-A-Video needs 10 min with GPU A100; MotionDirector needs 20 min with GPU A6000 (48G); while our approach needs about ~30 min with only one RTX 4090 (24G) (II) Exisiting approach like MotionDirector achieves a lower resolution for 384x384, rather than 512x512 in other methods. (III) We further validate our approach with Nvidia L20 (48G) which shows that our approach spends about ~20 min, which is on par with existing methods but achieves more compelling results (Tab.1 & Fig.3).

Dear Reviewer yeHN,

Hoping this message finds you well. Your comments and the provided insights are very valuable to our work. We will attach these analysis and additional experiments to our final version. As the rebuttal-discussion period is nearing its end, could you please review our response (especially in https://anonymous.4open.science/w/STIVE-PAGE-B4D4/) to see if it addresses your concerns? Your timely feedback will be extremely valuable to us. Could you read and let us know if there are more questions? We would be very grateful! Your decision is of utmost importance to us, and we earnestly hope that you will consider the contribution of this research to the field of one shot video editing. Thank you very much!

Best regards,

The Authors

We thank the reviewer for their feedback and for recognizing the effort we put into additional experiments and analysis. We appreciate the acknowledgment of our exploration of video model backbones and the analysis of the mentioned papers.

(1) On Novelty and Contribution: While we respect your perspective on the novelty of the methodology, we would like to reiterate that our approach addresses a unique and underexplored challenge in concept-driven one-shot video editing. Specifically:

(a) The selective fine-tuning of V projection weights in the attention mechanism is a targeted and efficient solution to integrate novel concepts while preserving consistency and stability.

(b) The use of concept pairs (text and video) introduces a new level of controllability, enabling precise embedding of specific target objects from a reference video into a source video, which purely text-based methods cannot achieve.

(c) Our method strikes a practical balance between computational efficiency and visual quality, a crucial consideration for real-world applications.

(2) Visual Quality and Computational Efficiency: We acknowledge the reviewer's concerns about performance in terms of visual quality and computational efficiency. The limitations in visual quality primarily stem from the inherent challenges of one-shot video editing and the need to handle diverse and arbitrary concepts. Despite this, our results show significant improvement over existing baselines, and we believe this demonstrates the effectiveness of our approach.

(3) Fine-tuning V Projection Weights: We appreciate the suggestion to delve deeper into why fine-tuning the V weights is efficient compared to tuning the entire model or other parts. The key insight lies in:

(a) Stability and Focused Adaptation: Fine-tuning V allows the model to integrate new feature representations into the pre-trained attention mechanism while minimizing disruptions to the pre-existing distribution, thereby enabling stable training.

Efficiency: By restricting updates to V, we reduce the number of trainable parameters, achieving a lower VRAM overhead without sacrificing performance.

Task-Specific Design: This selective tuning is particularly suited to our one-shot video editing task, where preserving the pre-trained model's knowledge while adapting to new concepts is crucial.

We will incorporate a more detailed analysis of these aspects in future revisions to strengthen our contribution claims. Thank you again for your valuable feedback. We deeply appreciate your constructive suggestions and will use them to further improve our work.

We thank Reviewer-oUvi for the comments. Reviewer-oUvi thinks "The two proposed methods intuitively make sense" but gives a reject rating due to questioning the core methodology ? We address Reviewer-oUvi's concerns bellow.

Reviewer-oUvi criticized our writing "needs improvement" -- for example, not clearly stating about the core proposed components.

Thanks. We kindly disagree with the comment. We already highlight our motivation and propose Concept-Augmented Textual Inversion and Dual Prior Supervision at Line 61-69. (I) For Concept-Augmented Textual Inversion, we enable one-shot flexible video editing based on external word embedding and target video (what we term concept prompt and concept video), we further advance the textual inversion tailored for concept-augmented video editing by addressing the under-fitting issue inside the original approach. (See lines 199-235) (II) We propose Dual Prior Supervision (SCAM and TCAM) to supervise the attention between word embedding and unrelated areas in video to maintain consistency. (See lines 236-273). (III) Both reviewers appreciate our draft "clear writing", "easy to follow". May we humbly ask for more specific details that support 'writing needs improvement'?

Novelty of our Concept-Augmented Textual Inversion (CATI)

Thanks. We would like to emphasize that the reviewer may have confused the distinction between our motivation and the specific implementation. We aim to flexibly edit source videos using concept videos as a key task setting, with our primary motivation being to enrich the latent space through one-shot concept videos to achieve more faithful inversion representations. We validate this improvement through ablation studies, as shown in Fig. 4. We deeply appreciate the reviewer's feedback and would like to clarify that the proposed CATI method is technically novel. While LoRA and Textual Inversion have been widely used separately in image style transfer tasks, to the best of our knowledge, no existing variant of textual inversion simultaneously employs LoRA tailored for video editing task. Existing approaches that do fine-tune the denoising network face significant challenges in one-shot concept video scenarios, such as overfitting and high memory requirements during training. Our approach addresses these limitations by leveraging LoRA modules to fine-tune attention value weights with a smaller learning rate, maintaining low VRAM overhead during tuning while preserving plug-and-play capabilities. As also appreciated by Reviewer-HLfd "The motivation is clear". May we kindly request citations if similar motivations exist?

Novelty of Dual Prior Supervision (mask generation by thresholding the activated attention, specifying areas corresponding to certain words through hard thresholding )

Thanks. (I) We would like to clarify that there appears to be a misunderstanding regarding the mask generation process. The mask is actually obtained through a pretrained OWL-ViT model, not by thresholding the activated attention, as detailed in Lines 252-254 of our paper. We appreciate the mention of MasaCtrl, which is indeed an excellent work for consistent image synthesis and editing using mutual self-attention control. While previous works like Prompt-to-Prompt and FateZero have made valuable contributions with cross-attention control and self-attention blending control methods, our method serves as a supervision mechanism designed to reduce inconsistencies in non-target areas before and after video editing, addressing the low consistency in existing approaches, especially in editing source video with concept video (As illustrated in Tab.1 in our paper. The reviewer ZGWC also mentioned that our SCAM and TCAM is novel). (II) Noted that Dual Prior Supervision does not specify areas corresponding to certain words through hard thresholding. Instead, the SCAM and TCAM losses are designed to optimize only the attention probability between unrelated areas and certain words toward zero, without imposing constraints on the probability between related areas and the corresponding words. This mechanism is formally described in Eq.(5) and Eq.(6) of our paper.

Thank you for your detailed response and for pointing out some parts I had missed. Regarding the Dual Prior Supervision, the reviewer's response has convinced me and alleviated my concerns about this part.

First, I would like to clarify my understanding to ensure we are on the same page: CATI utilizes textual inversion, a technique commonly employed in T2I models, to internalize the concept of a user-provided video for video editing. In doing so, instead of merely applying textual inversion, the authors incorporated LoRA to address the issue of underfitting. (Please correct me if my understanding is wrong.)

My concerns are: (1) Using LoRA to avoid underfitting is a method commonly employed in T2I models, and thus does not offer new technical contributions when adapting to video editing (https://github.com/cloneofsimo/lora); (2) Employing textual inversion to internalize a concept video is a straightforward approach, and it may be challenging to consider this innovative. There might not be substantial technical insights to gain from this aspect.

I do acknowledge the authors' conceptual contribution of utilizing concept videos, irrespective of the implementations. However, as reviewers yeHN, G4M5, ZGWC, and I have similarly pointed out, there is concern that this method may lack sufficient technical contribution. The notable difference in the authors' proposed LoRA+textual inversion tailored for video editing appears to be the selective training of the V projection LoRA. It would be beneficial to emphasize these aspects in the paper. Overall, CATI leans more towards a conceptual contribution rather than specific architectural design choices. However, I feel that the current manuscript could improve in clearly articulating these contributions. For readers not deeply engaged with this field, the introduction may seem somewhat vague. For instance, terms like "concept template" (Line 71) and phrases such as "constrained by the size of the latent space" (Line 59) required me to do multiple readings to comprehend. Does the "latent space" mentioned refer to the space of text embedding features? I recommend that the authors include a figure prior to the current Figure 1 to better highlight the conceptual novelty distinguishing this work from existing approaches. It would be also beneficial to include detailed explanations of the newly added inputs: concept video and concept prompt and how they relate to existing inputs: the source video, source prompt, and target prompt, throughout the introduction and method sections.

Given that my concerns regarding the Dual Prior Supervision section have been addressed, I am revising my score to a borderline reject.

Thank you very much for your thoughtful and constructive feedback. We greatly appreciate your patience and the time you've taken to carefully review our work. We are pleased to hear that our clarifications regarding the Dual Prior Supervision have alleviated your concerns, and we will address the points you raised below.

About lack sufficient technical contribution.

We deeply value the reviewer's feedback, but we believe CATI we proposed has technical contribution. Compared to our method, the conventional Textual Inversion approach lacks novel object information in arbitrary concept videos which causes word embeddings obtained through inversion having insufficient fidelity in describing target objects. Directly porting Textual Inversion to one-shot video editing scenario struggles from (always failed) generating videos for novel provided concept pairs. Our approach addresses these limitations by utilizing cutting-edge LoRA modules to fine-tune attention value weights (We latter explain why we only tune V), effectively maintaining the advantages of low VRAM overhead during tuning while preserving play-and-plug capabilities.

As for V projection LoRA for training, our goal is to perform inversion while integrating novel object information in arbitrary concept videos for one-shot video editing task. Textual inversion process relies on the pre-trained denoising-net established text-image attention probability distribution to achieve accurate target representation. In this context, fine-tuning only the V weights instead of Q and K allows new feature representations to be directly integrated while suppressing changes towards the pre-trained attention distribution to enable stable training during the early stages.

"latent space"

Thanks. In our work, the term "latent space" does not refer to the text embedding feature space. Instead, it refers to the latent space of the latent diffusion model in [1], specifically the spatial-level lower-dimensional latent representation produced by the variational autoencoder during the generative process. This latent space is a compact representation of the input data and serves as the working space for the diffusion model's operations. When we mention "constrained by the size of the latent space" we are referring to the fact that the capacity of the diffusion model's latent space is inherently limited by the scale and diversity of the image samples used during pre-training. This limitation (see explanations for previous question) affects the representation power of the latent space, consequently, the quality of the inversion results. Relying solely on Textual Inversion to obtain a word embedding from concept an image or video often leads to insufficient representation capabilities, especially for samples far from dataset for pre-training.

[1] Rombach R, Blattmann A, Lorenz D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 10684-10695.

"concept template" & Details of how concept video and concept prompt relate to existing inputs.

Good questions! The template here means we could change the source video object with several different concept videos coupled with specified objects to enable more diverse and flexible one shot video editing. For example,

• source prompt : "a man rides a wooden skateboard on the handrail of the staircase with arms outstretched"
• concept prompt - I : "a OPTIMUS is squatting on a wooden floor with its back to a white wall"
• target prompt - I : "a OPTIMUS rides a wooden skateboard on the handrail of the staircase with arms outstretched"
• concept prompt - II: "NEO is outstretching his arms"
• target prompt - II : "a NEO rides a wooden skateboard on the handrail of the staircase with arms outstretched"
• concept template: "a 【CONCEPT】 rides a wooden skateboard on the handrail of the staircase with arms outstretched"

For current source video and prompt, when employing different target pairs we can obtain diverse editing results based on concept tamplate. For more details, please refer to Fig.3(setting I) and Fig.13.