Consistency Verification for Detecting AI-Generated Images

Weakness 4: The experiment results of RIGID are not the same, but deteriorated from the original paper, especially in the ImageNet dataset. I wonder why such a discrepancy occurs, given that the performance gap between the proposed method and RIGID is not so large. This is more puzzling since the performance of RIGID reported in this paper should be improved by using larger $n = 20$. I hope the elaboration on this result will be beneficial to resolve my concerns.

Response:

The authors of RIGID have not yet released their code, so we were unable to obtain information in their experimental section about the way their data is preprocessed and the strength of the Gaussian noise they added. Therefore, we used the same data preprocessing for their method as ours and set the Gaussian noise strength to 0.1 based on the results of the ablation experiments in their paper. These differences may have contributed to the performance gap.

Weakness 5: I did not quite understand of the data processing procedure of the GenImage dataset on Page 15. I feel like the authors cropped the AI-generated images in 224 by 224 dimensions, but treated ImageNet data differently by resizing first, and then cropping it to 224 by 224 size. Did I understand it correctly? If it is, why the processing method is different in AI-generated and real images?

Response:

Thanks for your comments. Your understanding is correct. In the original GenImage dataset, the natural images are all saved in jpg format, while the generated images are all saved in png format, and this unwanted bias will result in unrealistic detection performance, which is discussed in AEROBLADE [ref2]. Therefore, we follow [ref3] to convert all the natural images to png format and pre-scaled the images to 256*256. Since the generated images are already in png format, we don't do anything with them beforehand. We will include these descriptions in our revision.

Weakness 6: I am curious how AEROBLADE will perform in the GenImage dataset. (The table has typos by the way)

Response:

Thanks for the heads up, we've corrected it in the revised version. For AEROBLADE, since GenImage is too large, we don't have enough GPUs to run AEROBLADE on the full dataset at this moment. Therefore, for each class of generative models, we randomly pick 10,000 natural images and 10,000 generated images to test its performance. The results are as follows:

Weakness 7: While this is minor, I would like to see the runtime analysis of the proposed method, given that the paper utilizes lots of ensembles.

Response:

Thanks for your kind suggestion. As shown in the table below, we count the time used to detect 100 images. When using 20 ensembles, ConV takes longer detection time than AEROBLADE. Of course, this issue can be addressed by parallelization or reducing the number of ensembles. When reducing the number of ensembles, the performance suffers a slight degradation, as shown in Figure 4.

Reference:

[ref1] RIGID: A Training-Free and Model-Agnostic Framework for Robust AI-Generated Image Detection. arxiv, 2024.

[ref2]: AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error. CVPR, 2024.

[ref3]: Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. NeurIPS, 2024.

We would like to thank Reviewer qU6e for taking the time to review our work. Please see our detailed responses to your constructive comments below.

Weakness 1: I feel the novelty of the paper is weak. The paper borrows lots of ideas from the RIGID. RIGID measures cosine similarity between the DINOv2 representation of the original image and the DINOv2 representation of the deviated image through Gaussian Noise Augmentation. This paper does the same except changing cosine similarity to l2 distance, and Gaussian augmentation to training augmentations (e.g. Gaussian Blur, Colorjitter). Hence, I feel like this paper is just an ablation study of the RIGID paper, and the contributions should go to the original paper (e.g. applying DINOv2, and proposing a similar consistency function).

Response:

We respect your understanding of our method through your perspective. Please let us list points to highlight the differences from the mentioned RIGID.

First, our method is based on the derived orthogonal principle, which ensures the generalization performance of our method. In contrast, RIGID is based on the empirical observations, leading potential risk of RIGID. Namely, the performance of RIGID could drastically degrade when generated images are perturbed with Gaussian noise. Detailed results and discussions can be found in the responses to obBE’s Weakness 2.

Second, the realization of our orthogonal principal is not merely focusing the similarity between images and their transformed versions, as shown in the responses to Reviewer t4Ze [Question 1]. In contrast, as mentioned in your comments, RIGID mainly focuses on the similarity between features of images and images with Gaussian noise.

Third, as mentioned in your comments, we compared our method with RIGID. This is because we appreciate the contribution of RIGID due to its simple yet effective training-free framework. We believe it would be a standard baseline in our community, so we set it as our baseline method.

Weakness 2: Experiment results also feel weak. The proposed method does not outperform RIGID in all cases. Aligned to 1), I feel like this paper should at least outperform RIGID in all experiments, not only on the average but also in all AI-generated data. Maybe more deeper analysis will be helpful.

Response:

Due to the diversity of generative models, the various generated images may have different properties, and it is challenging for a single method to outperform recent SOTA work in all cases. Nevertheless, ConV outperforms RIGID on average across various benchmarks.

Weakness 3: Section 2 is hard to read. 2.1~2.3 seems to overexplain general ideas of manifold hypothesis, which is not new. In my opinion, this is just a longer extraction of "This model is trained on the real data so it will show robust output on the real data when the training data augmentation is applied". The final consistency term form has been applied to various binary classification tasks ([ref2],[ref3]). Maybe a deeper explanation of why this explanation is necessary will be helpful.

Response:

We apologize for any confusion. Accordingly, we would like to highlight the main points in Section 2.

In Sec. 2.1, we give the general ideas of manifold hypothesis. In Sec. 2.2, we give the orthogonal principal, namely, the introduced two functions can detect generated images through the orthogonal principal: gradients of these two functions should lie within two mutually orthogonal spaces. In Sec. 2.3, we give a realization of the orthogonal principal based on the manifold hypothesis discussed in Sec. 2.1.

We agree that OOD detection provides many outstanding methodologies, such as the consistency perspective [ref2, ref3]. However, researchers can able to draw inspirations from OOD detection. For instance, post-hoc OOD detection methods usually leverage pretrained models to detect OOD data, which will not be the reason to weaken the contribution of RIGID, namely, RIGID is an outstanding post-hoc approach to detect generated images using a pretrained model.

We would like to thank Reviewer t4Ze for taking the time to review our work. We appreciate your recognition of well-presented mathematical intuition, novelty, and effectiveness of our proposed method. Please see our detailed responses to your constructive comments below.

Weakness 1: A formal proof of the convergence of ConV’s generalization risk could strengthen the theoretical grounding.

Response:

We agree that a formal generalization risk could strengthen the theoretical contribution. It is challenging to develop a novel theory for a novel framework due to the limited time. Thus, we leave it as our future work.

Weakness 2: To avoid potential confusion, you should revise the description to clarify that subtle differences still make AI-generated and natural images distinguishable.

Response:

Thanks for your kind suggestion. We have revised it as follows:

In particular, for a given generated image $x_g$ , even if it captures similar content to a nature image $x_n$ , humans know they are distinguishable in certain ways.

Weakness 3: As shown in Table 6, it appears that ConV is sensitive to the initialization function. ConV instantiated by CLIP performs much worse than DinoV2. Is there an approach to overcome this limitation?

Response:

We sincerely appreciate your insightful question. To overcome this limitation, we revisit our methodology, i.e., verifying the consistency between outputs of two functions.

As shown in Eq. (13), we derive the cosine similarity metric between image features from a self-supervised learning objective function. However, CLIP employs a different objective function, namely, calculating the similarity between text and image features. Thus, the proposed cosine similarity between image features may not be a good realization of these two functions’ output, limiting the generalization capability for generated image detection.

Inspired by your valuable question, we conjecture the difference between the projection of the visual features of the original image and the visual features of the transformed image on their corresponding text features would be a good metric. The reason is as follows.

The function to calculate the similarity between text and image features can be regarded as function $f_1(x, t)$. Thus, we should calculate the difference in inter-modality similarity rather than the similarity between original and transformed images. To verify the point, we conduct experiments using the corrected realization of $f_1$ and $f_2$, i.e., a corrected metric to verify the consistency. The results below show that the modified approach outperforms the original metric.

As shown in the following table, using the modified metric for detection greatly improves the performance of CLIP-based methods model, achieving performance comparable with Dinov2-based methods.

We will incorporate the above analyses, results, and discussions into the revision. Inspired by your valuable question, we believe our work provides a novel approach to calculate the difference between two functions, without the focuses on the difference in similarities between two image features. Thanks again for your insightful and constructive question!

Dear Reviewer #t4Ze,

Thank you very much for your time and valuable feedback.

Here is a summary of our response for your convenience:

(1) Issues regarding ConV's generalization risk: Developing a novel theory for a new framework is challenging due to the limited time available. We will address this in future work.

(2) Issues related to clarity in wording: Following your suggestion, we have revised the sentence as follows:

In particular, for a given generated image $x_g$, even if it captures similar content to a nature image $x_n$, humans know they are distinguishable in certain ways.

(3) Issues regarding ConV's performance on CLIP: We have successfully improved the performance of ConV on CLIP by calculating the difference in inter-modality similarity, rather than the similarity between original and transformed images.

We understand that you're busy, but as the window for responding and revising the paper is closing, would you mind reviewing our response and confirming if you have any further questions? We would be happy to provide additional answers or revisions if needed.

Best regards and thanks, Authors of #8640

Weakness 3: The paper does not give specific details of the data augmentation used, including the parameters of the data augmentation used, and whether a single augmentation or a combination of multiple augmentations is used.

Response:

Thanks for your comments. We follow the data augmentation strategy used when training DINOv2 with a combination of HorizontalFlip, ColorJitter, and GaussianBlur. For ColorJitter, brightness, contrast, saturation, and hue are randomly adjusted with a factor in the ranges of [0.88,1.12],[0.88,1.12],[0.94,1.06], and[0.97,1.03], respectively. For GaussianBlur, the kernel size is set to 9*9, and the variance is randomly selected in [0.7,1]. We'll add this detailed description to the revision.

Weakness 4: The paper only tested on DinoV2 and not on more (self-supervised) models. Does this mean that the scope of application of this method is limited?

Response:

We apologize for the misunderstanding. TWO models, DINOv2 (self-supervised model) and CLIP (multimodal model), are used to evaluate the effectiveness of our method. Inspired by your valuable questions, we conduct experiments using the MoCo, SwAV and DINO. The results are as follows. These results show that our method can be applied to various backbones. We will add these results to the revision.

Reference:

[ref1] Tangent prop-a formalism for specifying selected invariances in an adaptive network. NIPS, 1991.

[ref2] RIGID: A Training-Free and Model-Agnostic Framework for Robust AI-Generated Image Detection. arxiv, 30 May 2024.

We thank Reviewer obBE for taking the time to review our work. Please see our detailed responses to your comments below.

Weakness 1: There is no explanation for why the augmentations during DinoV2 training are orthogonal to its standard output? Moreover, the theoretical analysis in the paper does not provide any guidance for the choice of augmentations.

Response:

We apologize for the misunderstanding. Our focus on orthogonality is mainly about the gradients of the introduced two functions $f_1$ and $f_2$. Namely, the gradient of $f_1$ w.r.t the input is orthogonal to the local tangent space, while the gradient of $f_2$ w.r.t the input lies in the local tangent space (c.f. Eqs. (8) and (11)). The reason of why the gradient of $f_2$ lies in the local tangent space is that we realize the function $h$ of $f_2$ using the transformation along local data manifold. Our theoretical analysis shows that we can realize the function $h$ using transformation along local data manifold. One simple yet effect approach is to employ data augmentations used during training phase. This is because data augmentations are one approach to move data point along local tangent space [ref1].

Weakness 2: The method proposed in this paper is based on Baseline RIGID, but it only replaces Gaussian noise with training enhancements, without much insightful design. In addition, it does not explain why these enhancements are better than Gaussian noise.

Response:

We apologize for the misunderstanding.

First, our theoretical analysis shows that the gradients of the introduced two functions need to lie within two mutually orthogonal subspaces. Thus, we propose to leverage the transformation along its data manifold to realize the function $h$. It is known that data augmentation can be regarded as transformation along data manifold. Thus, we propose to use data augmentations to realize the function $h$. In contrast, RIGID’s theoretical analysis shows that RIGID mainly focuses on the difference in gradient norm between natural and generated images for detection.

Second, our method is not based on [ref2]. As shown in the response to Reviewer t4Ze [Question 1], replacing the cosine similarity between images with consistency of projections of original image features and transformed image features on text features can significantly improve the performance on CLIP. This highlights the novelty and contribution of the proposed consistency verification framework. Namely, our method does not focus on the difference in similarities between an image and its transformed version. In contrast, our work focuses on verifying the inconsistency between two functions that have gradients mutually orthogonal to each other.

Third, our method significantly outperforms the mentioned RIGID, especially on images generated by the inaccessible generative model Sora. This is based on our orthogonal principal. In contrast, RIGID is based on the observed sensitivity of the generated images to noise, leading to potential risks. For instance, users can evade detection by adding noise to the generated image. As shown in the table below, when noise is added to a generated image, RIGID can easily be tricked into identifying it as a natural image.

Fourth, we would like to highlight that the consistency verification is the insightful design of our work. Specifically, we can introduce various metrics and functions to calculate the inconsistency. For instance, designing metric for specific models, as shown in our response to Reviewer t4Ze [Question 1]. In addition, we can introduce various approaches to construct different these two functions. All these directions are based on our orthogonal principal, which is not given in the mentioned work RIGID.

Dear Reviewer #obBE,

Thank you very much for your time and valuable feedback.

Here is a summary of our response for your convenience:

(1) Issues regarding theoretical justification: Our focus of orthogonality is mainly about the gradients of the introduced two functions $f_1$ and $f_2$. Namely, the gradient of $f_1$w.r.t the input is orthogonal to the local tangent space while the gradient of $f_2$ w.r.t the input lies in the local tangent space (c.f. Eqs. (8) and (11)). The reason of why the gradient of $f_2$ lies in the local tangent space is that we realize the function $h$ of $f_2$ using the transformation along local data manifold.

Our theoretical analysis shows that we can realize the function $h$ using transformation along local data manifold. One simple yet effect approach is to employ data augmentations used during training phase. This is because data augmentations is one approach to move data point along local tangent space.

(2) Novelty Issue: The main difference between ConV and RIGID is that we focus on designing a pair of functions to verify consistency, where the natural image exhibits high consistency on these functions, and the generated image shows low consistency. our theoretical analysis shows that the gradients of the introduced two functions need to lie within two mutually orthogonal subspaces. Based on this analysis, we can design various metrics to compute the consistency for specific models. In contrast, RIGID focuses on the difference in gradient norm between natural and generated images for detection.

(3) Issues of implementation details: We follow the data augmentation strategy used when training DINOv2 with a combination of HorizontalFlip, ColorJitter, and GaussianBlur. We've added the details to the revision.

(4) Issues of backbones: Following your suggestion, we have implemented ConV on additional backbones. The experimental results demonstrate the universality of ConV.

We understand that you're busy, but as the window for responding and revising the paper is closing, would you mind reviewing our response and confirming if you have any further questions? We would be happy to provide additional answers or revisions if needed.

Best regards and thanks, Authors of #8640