Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation

We thank the reviewer for their positive feedback. We are glad that they found our paper “easy to read”, with a “simple idea” and containing an “extensive ablation study”.

In the following, we address each question individually.

Q1: Can you place a link to section D of the appendix in the main paper in section 4?

Thank you for the suggestion! Yes, we will include this link in the paper and better connect the main paper to the appendix.

Q2: A graphic with the full prediction model (incl link to the backbones discussed) in the supplement would be helpful. That would help to consolidate the information which is a bit scattered in the main paper.

Yes, we will add such a graphic in the supplemental material and in the main paper, and describe more clearly the information about the models used.

We thank the reviewer for their positive feedback on our paper. We are glad that they found our methodology “rigorous, clearly motivated by a thorough analysis of the forensic artifacts”, our wavelet-based augmentation strategy “novel within the context of forensic-oriented video detection”, our experiments “comprehensive and well-structured, including recent generators and established benchmarks”, our paper “clearly written, with logical progression from motivation through method, results, and discussion”, that our “approach directly addresses a significant limitation in current forensic video detection research” and that our “results demonstrate meaningful improvements in generalization”.

In the following, we address each question individually.

Q1: Temporal Artifacts: Given the promising results, how could the proposed augmentation strategy be extended or adapted to explicitly include temporal forensic artifacts, which could further enhance detection performance?

Our augmentation strategy is designed to prevent the detector from focusing on horizontal and vertical spatial frequencies where compression artifacts may overlap with forensic cues. Similarly, this approach can be extended to the temporal domain, encouraging the detector to exploit temporal forensic artifacts. One possible direction is to replace the low temporal frequencies of the fake video with those from the real video during training, while preserving the mid-to-high temporal frequencies of the fake content. We plan to take this direction in our future work as well as better understand the forensic artifacts in the temporal domain, since we believe that the replacement strategy should be focused on specific temporal frequencies. However, a key challenge in this respect is to identify a 3D backbone visual encoder capable of matching the performance of 2D models like DINOv2. As we can see from Table 2 the performance of the latter is significantly better than that of the 3D Hiera video model.

Q2: Future Generators: How robust do you expect your method to be against significantly different generative paradigms that may emerge in the future? What adaptations, if any, would be necessary to maintain high generalization?

The forensic artifacts that we have analyzed in our work are caused by the upsampling operation in the network’s architecture (resulting in peaks in the frequency domain), which has also been shown in prior work [A,B]. This is caused by the inability of the generator to perfectly reproduce mid frequencies of the video signal as also shown in [C,D]. If new generative paradigms still include upsampling, then such traces will likely be present (maybe only in different positions). This is also what we observed with autoregressive models that are different from diffusion models. Despite having a different generative paradigm, they still present similar artifacts (see Figure 2 of the main paper). However, it is not easy to predict what will happen in the future. Hence if the generative paradigm will completely change, then the method would probably require re-training on a more recent generator. Even so, we believe that the discussion on the compression artifacts would still be relevant and hence our augmentation strategy could still be applied as is.

References:
[A] Zhang et al. Detecting and Simulating Artifacts in GAN Fake Images, WIFS 2019.
[B] Vahdati et al. Beyond deepfake images: Detecting AI-generated videos, CVPRW 2024.
[C] Dzanic et al. Fourier spectrum discrepancies in deep network generated images, NeurIPS 2020.
[D] Corvi et al. Intriguing properties of synthetic images: from generative adversarial networks to diffusion models, CVPRW 2023.

Q3: Computational Efficiency: While wavelet decomposition is mentioned as computationally efficient, could you provide specific benchmarking or computational complexity analyses comparing this augmentation strategy to simpler augmentations such as MixUp or CutMix?

While wavelet decomposition is computationally efficient, it is slower than operations such as MixUp or CutMix. More specifically, the wavelet augmentation (with three decomposition levels) requires about 21N FLOPs (floating-point operations), while MixUp requires only 3N FLOPs, where N is the number of pixels. However, the implementation of the wavelet decomposition using convolution reduces the execution time substantially on parallel hardware architectures (GPUs) with an overall execution time of 18.6 ms for batch compared to 5.2 ms for MixUp on a A6000 Nvidia Quadro. This increase in execution training time is compensated, however, by an improvement in terms of accuracy (+3% and +5%, Table 1).

Q4: Dataset Diversity: Do you anticipate similar performance benefits when applying your augmentation strategy to significantly smaller or differently structured training datasets?

We conducted two additional experiments: (A) we used only 20% of our training set and (B) we trained on videos generated with a different synthetic generator (CogVideoX). Results for both experiments are presented below. In the first case we observe that our proposed augmentation strategy is still able to provide an advantage on a significantly smaller training set (+15% in terms of balanced accuracy) (second row).

Table A: 20% of training set from Pyramid flow.

In the second experiment we used CogvideoX, instead of Pyramid Flow, during training. In this case too, we observe a significant advantage of the proposed augmentation strategy (+5% in terms of AUC and +13% in terms of balanced accuracy) (second row).

Table B: Using CogVideoX as the training set.

We thank the reviewer for their positive feedback on our paper. We are glad that they found our “submission technically sound”, the “claims well-supported by experimental results”, our “submission clearly written and easy to follow” and that our “results are impactful for the community”.

In the following, we address each question individually.

Q1: The paper makes valuable contributions, including a new dataset and a promising method. However, neither the code nor the dataset has been made available. The authors are strongly encouraged to release both the code and dataset to enable transparent evaluation, reproducibility, and faster progress in the rapidly evolving field of forensic video detection.

We plan to publicly release both the dataset that we created with new synthetic generators and the code of our method to guarantee reproducibility, and to help advance the research field on video forensics.

Q2: a. In lines 186–193, the authors state that replacing the low-frequency bands with those from the real counterpart forces the detector to focus on mid-high frequency features. However, how can we confidently claim that the detector will focus on mid-high frequencies? It is possible that the detector still learns from the low-frequency content provided by the real counterpart, potentially leading to misleading or incorrect detection. Do the authors have empirical evidence or observations supporting this assumption?

To gain better insights into this issue we conducted the following additional experiment: at test time we modified the real videos and injected the mid-high frequencies of fully synthetic videos into them. The results of our detector with this test set are presented below. We observe that the results in terms of AUC and balanced accuracy (bAcc) get worse (second row). Moreover, in terms of the True Negative Rate (TNR), which is the probability that real videos are correctly classified as real, it goes from 98.7 to 0.2 (on average), which clearly shows that the detector inverts its decision. This highlights the importance of mid-high frequencies to establish if a video is fake or not.

**real videos with mid-high frequencies of fully-generated videos

Q3: The proposed method appears to heavily rely on the reconstruction quality of the Pyramid Flow model. While the paper mentions that this model generally reconstructs videos without visible artifacts, what happens if it fails in certain cases? Would such failure compromise the robustness or generalizability of the detector trained using this reconstruction-based augmentation?

Even though our method is trained on high-quality synthetic videos generated by the Pyramid Flow model (with a quality score above 84% on the VBench Leaderboard [30]), it is still able to detect videos with visible artifacts. This is evidenced by the fact that for videos generated by the ModelScope model that exhibit visible artifacts (with a quality score below 76% on the same benchmark [30]), our method achieves a high accuracy of 97.1%. Furthermore, overall, from our experiments we were not able to observe a different behaviour between older and newer synthetic generators (the former containing more visible artifacts) (see Table 3 and 4 of our paper). Our conjecture is that the detector is mostly focused on low level traces and is not very much influenced by semantic inconsistencies.

We thank the reviewer for their positive feedback on our paper. We are glad that they found that “it addresses an important problem”, its motivation is “very clear, connected to implementation”, and “related with the literature”, that our “approach is technically sound and results are promising”, our “observations on the relation between compression and generative fingerprints are very interesting” and that “training only on one generator and generalizing to 15 generators is an excellent evaluation”.

In the following, we address each question individually.

Q1: "generation-specific artifacts" vs. "artifacts introduced by specific generators".

Thank you for pointing this out. We will make clearer the distinction between "generation-specific artifacts", which we consider to be traces related to the generation process, that are distinct from "artifacts introduced by specific generators", e.g., certain semantic visible cues, such as the lack of perspective or temporal consistency in the generated videos.

Q2: Denoiser used in visualization.

We used the denoiser only to extract the forensic fingerprints and to motivate our method. However, we do not use a denoiser in our detector, which is only based on the wavelet transform to implement our frequency-based augmentation strategy. We will make this clearer in the paper and give more details of the denoiser.

Q3: Training on 15 generators.

This is an interesting question. While including more generators might seem to improve performance, the answer is not straightforward. A key issue is which generators to include during training. For example, DeMamba [12] was trained on 10 generators (from 2023–2024) and performs well on older models but struggles to generalize to newer ones (see Table 3). This suggests that both the choice and the number of generators remain open research questions.

It is also not feasible for us to apply our data augmentation procedure to many generators that require reconstruction with the same autoencoder used during generation. Nevertheless, as one datapoint, we conducted an experiment, wherein, we compared the model trained using two generators (Pyramid Flow (P) + CogVideoX (C)) with the same model trained using one single generator (P or C) and their corresponding versions with our wavelet-based augmentation. In the following table we show both AUC and balanced accuracy (bACC). We observe that using one single generator with its simpler design and our augmentation gives better results than including two different generators (92.5 and 95.3 vs 87.3 in terms of bACC). Arguably, adding more generators may potentially change these results. Nevertheless, it shows one quantified datapoint.

Q4: Compression robustness.

We compressed the test dataset used in our ablation study (both real and fake videos) with H.264 at different quality levels by varying the Constant Rate Factor (CRF). Results using different wavelet decomposition levels are presented in the Table below and show that our method is robust to different compression levels with AUC always remaining above 80%.

The accuracy gets significantly lower when the quality decreases (around 60% in the last row, CRF 32) and this clearly indicates the need for a calibration procedure for low-quality compressed videos to estimate the optimal threshold value. By using only 10 compressed videos from the validation set for threshold calibration, we noted an increase in accuracy from 60% to 76%. We will include these results in the revised version of our paper.

Q5: Augmenting SoTA training data.

We understand this critique. However, it is not always obvious as to how to perform this analysis, since the training sets of current detectors can include generators whose architectures are not disclosed (e.g., Moonvalley, Pika), which means that we cannot apply our procedure that requires data reconstruction with the same autoencoder used during generation. In addition, some prior methods employ high-level semantic traces for detection. Hence our augmentation, which is focused on low-level traces makes less sense in their context.

Nevertheless, we re-trained the method DMID [18] on our dataset and obtained an average improvement of +1.5 in AUC and +1.9 in Accuracy on the more recent generators (2024-25 years) by including our augmentation strategy. We will add more such comparisons with other prior datasets in the revised version of our paper.

Q6: Other frequency-based detectors.

There are indeed other methods that leverage frequency domain analysis to design an effective forensic detector. Some approaches aim at forcing the detector to focus on high-frequency information [A, B], while others exploit discrepancies in the whole frequency spectrum that can be achieved, e.g., by concatenating frequency-based components in the DCT domain [C]. A more recent work is based on the observation that the reconstruction error exhibits stronger mid-frequency components for real images versus for generated images [D]. In our approach, instead, we guide the detector to focus on diagonal mid-high frequencies, as video compression artifacts overlap with forensic cues in vertical/horizontal directions. Unlike [E], which uses frequency-domain Mixup without targeting model-specific cues, our method focuses on artifacts that persist after compression.

We will add a paragraph in the Related Work section of the paper where we will explicitly make clear the differences between such approaches and our method. Furthermore, we will add comparisons with those works whose code is publicly available. We already performed some comparisons. Results are presented below in terms of AUC/bAcc both on their training data and by re-training on Pyramid Flow. Our method guarantees better performance than these detectors even when trained on the same dataset.

References:
[A] Tan et al. Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning, AAAI 2024
[B] Wang et al. Dynamic Graph Learning with Content-guided Spatial-Frequency Relation Reasoning for Deepfake Detection, CVPR 2023
[C] Qian et al. Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues, ECCV 2020
[D] Chu et al. FIRE: Robust Detection of Diffusion-Generated Images via Frequency-Guided Reconstruction Error, CVPR 2025
[E] Kashiani et al. FreqDebias: Towards Generalizable Deepfake Detection via Consistency-Driven Frequency Debiasing, CVPR 2025

Q7: More in-depth analysis.

To support the conjecture that the mid-high diagonal frequencies are more discriminative, we conducted a further analysis in Section B of the Appendix, where we compare real and reconstructed videos in the frequency domain by computing the distance defined in equation 7 (see supplemental material). A higher value for this distance indicates a greater difference between the real videos and the reconstructed ones, and therefore a more discriminative capability of the corresponding frequency. For mid-high diagonal frequencies the values concentrate around 89%, while for horizontal and vertical frequencies it is equal to 11%, indicating that the former frequencies are more discriminative.

To further compare to the established frequency domain research, we extracted mid-frequency and high-frequency content, which is mostly used in related work for discrimination [A,D]. In such regions the distance values concentrate around 55% and 68%, respectively. As a conclusion, the difference between real and fake videos is more pronounced at mid-high diagonal frequencies.

Q8: Writing and organization.

We will improve the algorithmic description of our method; add all suggested papers in the Related Work Section and change the order of the results.

Q9: Deepfake-eval-2024 dataset.

We ran our detector on the fully synthetic videos, obtaining an AUC of 82.4%.

Q10: Size of training/validation sets are significantly low.

This is true, however we do not see this as a weakness since our augmentation helps to compensate for that (this is also confirmed by the experiment we conducted in response to Q3).