Resolution Attack: Exploiting Image Compression to Deceive Deep Neural Networks

Weakness 3: (minor, experimental) I suggest including experiments under defensive settings to evaluate whether these adversarial samples can still successfully confuse classifiers when facing adversarial preprocessing (e.g., different resolution compression methods or adversarial purification).

We thank the reviewer for the suggestion. Specifically, we tested our RAS attack with additional compression techniques, including bilinear interpolation (INTER_LINEAR) and antialiasing (ANTIALIAS), alongside the default bicubic interpolation. Results are as follows:

These results indicate that the choice of compression method has minimal impact on attack success rates, because our attacks are semantics-driven and less sensitive to pixel-level transformations.

We apologize for not conducting adversarial purification experiments due to time constraints. However, we are optimistic about the experimental results: current purification methods, targeted for traditional pixel-level attacks, typically embed purification into the denoising process, neutralizing adversarial perturbations and Gaussian noise simultaneously. While our method compose semantic-level manipulation which is out of the scope of current purification methods. We will explore it as well as the effective defense method specially tailored for our proposed Resolution Attack for future work.

Weakness 4: (minor discussion) Given the potential risks associated with this work, I also recommend discussing the societal impacts and potential defence strategies at the end of the paper.

We appreciate the reviewer’s recommendation to discuss societal implications and defense strategies. In the revised paper (Appendix G), we have added a detailed discussion. Our method is designed to explore vulnerabilities in machine learning classifiers and assess robustness by generating dual-representation images. Though our approach is under the risk of malicious purpose, our generated image can be employed to fine-tune existing classifiers, further improving their robustness. Besides, the generated images can also be detected by deepfake detectors.

Weakness 2: (major, experimental) Although the effectiveness of the attack method on deep classifiers has been well demonstrated, all the models used are CNN-based classifiers with relatively low input resolution (224×224). In contrast, I am more interested in whether these adversarial samples can disrupt existing vision-language models (VLMs, e.g., CLIP, BLIP-2, LLAVA, etc.), which possess more excellent generalization capabilities than task-specific models. I think adding some evaluation results on VLMs, such as zero-shot classification or even image captioning and VQA, would significantly enhance the assessment of this method's value.

Thanks for your valuable insights. As you have pointed out, our method focus on the semantic of the images and is both model-agnostic and task-agnostic. To demonstrate this, we conduct experiments across various architectures (ViTs, FPN) and various tasks (VQA, Image Caption, Zero-Shot Classification). The results are shown in the following table:

 Additional quantitative results of the Resolution Attack

 Additional quantitative results of the Resolution Attack with Source image

The results demonstrate that our RA attack achieve over 90% success rates in unlabeled attacks on Blip2 and LLAVA 1.5. Similarly, in RAS attack, these models also exhibit strong attack susceptibility. The results demonstrate that our generated images can effectively attack various models across various vision tasks. We have incorporated these experiments in Appendix E of in the revised paper.

We express our gratitude to the reviewers for their meticulous assessment and valuable suggestions. Your detailed feedback is paramount in refining our work, and we have undertaken measures to address the concerns.

Weakness 1: (minor, discussion) Does the author thoroughly investigate the problem being studied? Specifically, are there existing resolution-based attack methods, and if so, should they be considered baselines? I suggest the author provide a more in-depth discussion of related work.

We appreciate the reviewer’s insightful comment on the discussion of related work. To our best knowledge, we are the first to propose the Resolution Attack (RA), an attack that is different from current arts. In the revised paper (Appendix D), we have elaborated on existing generative adversarial approaches in latent space, such as those leveraging diffusion models to manipulate U-Net parameters or introduce learned perturbations in latent space. While these works focus on adversarial attacks in generative modeling, none explore the unique dual-representation mechanism proposed in our resolution attacks. By introducing resolution attacks, we aim to provide a foundational framework for future research in this direction, fostering deeper understanding and advancements in the field.

Thanks for your response. My concerns have been addressed so I decide to raise my confidence score.

Weakness 3: Could the proposed method is till effective for vision transformer? The authors should give the discussion.

Appreciate the insightful query. We have conducted our proposed Resolution Attacks on Transformer-based models (ViT-B and ViT-L). Besides, according to suggestion provided by Reviewer dpTx and 6DtR, we have evaluated the effectiveness on Feature Pyramid Network (FPN) methods (Faster RCNN and Mask RCNN) and various vision-language models, including image captioning models (BLIP), Visual Question Answering models (LLAVA) and zero-shot classification models (CLIP). The results are shown in the following tables:

 Additional quantitative results of the Resolution Attack

 Additional quantitative results of the Resolution Attack with Source image

The results further demonstrate that our proposed method achieves favorable attack success rates on ViTs. Besides, our method can well generalized to various models across different vision tasks, emphasizing the broader applicability and effectiveness of our proposed resolution attack framework. We have incorporated these experiments in the Appendix E.

We sincerely appreciate the insightful feedback from the reviewers, and we have diligently addressed the raised concerns to enhance the quality of our work.

Weakness 1: It seems the test images are important for the success of resolution attack, because when an image is compressed into a low-resolution image, the semantic content is easily changed. In the paper, the authors don’t give the test dataset, so I don’t know whether the proposed method is still effective for all images.

Thanks for this advice. We would like to note that the Resolution Attack, which is included in the proposed settings, generates images from scratch without the requirement of a clean test image. Here, we mainly want to detail the setting that require clean test images (Resolution Attack with Source image, RAS). As we have stated in the original submission (Lines 415-416), we collect 100 frontal images as the source images due to the reason that facial features are critical for semantic representation. Similarly, in the following experiments on human faces (“Resolution Attcaks as the Face Swapper”), we also adopt human face images with clear frontal face.

To further demonstrate that our method is not sensitive to the choice of the semantic of the source images, we evaluate our method in broader classes. Specifically, we leverage 10 additional classes including boat, bird, cat et.al. which are in the class set of Cifar10 dataset. We exclude the class ‘deer’ and ‘horse’ as they are not in the label sets of target classifiers. The results are presented below:

These results demonstrate that our proposed method achieves robust attack performance across most categories, highlighting its generalizability.

Weakness 2: The method should be compared with the compression method that randomly reduce the resolution of the test image. Compared with this baseline, the readers can better see the effectiveness of this attack.

We first clarify that while directly applying compression operation can cause misclassifications, they cannot achieve targeted labels, unlike our proposed Resolution Attack, which achieves targeted classifications.

Moreover, current classifiers exhibit a certain degree of robustness to compression. To demonstrate this, we conduct experiments to evaluate the accuracy of current classifiers for both compressed natural images and generated images which is generated by SD.

 Accuracy of Random Image Compression

For traditional convolutional networks (ResNet50, VGG19), compression led to a higher proportion of misclassified images. Notably, classes such as "otter," "bear," "shoe," and "guitar" were more prone to errors, often being misclassified into similar or adjacent categories (e.g., "otter" being classified as "weasel" or "marmot"). In contrast, vision transformers (e.g., ViT-L32, ViT-B32) were significantly more robust to compression, resulting in fewer misclassified images.

Weakness 2: Generating adversarial images from scratch has limited practical application. In practical scenarios, adversarial images are typically crafted by modifying uploaded clean images to influence predictions while maintaining visual similarity to the original.

Maybe there is a misunderstanding. First, our method also support crafting uploaded lean images (RAS, Section 3.2&4.3). Second, our goal is completely different from the adversarial examples: we aim to generate images with dual labels to influence predictions in both high and low resolution, while adversarial examples only aim to influence the prediction. Additionally, adversarial attacks often overfit specific model architectures or inductive biases of particular tasks, limiting their generalization. In contrast, resolution attacks naturally generalize across diverse architectures and visual tasks. Thus, the resolution attacks is more challenging than adversarial attacks. Below, we clarify the difference between resolution attacks and adversarial attacks in detail.

Comparison with Traditional Adversarial Attacks: Traditional adversarial attacks focus on modifying clean images minimally, aiming to deceive classifiers without altering the image's apparent semantics. In contrast, our resolution attacks generates images with dual semantics. This innovative semantic duality extends beyond merely flipping labels, as it introduces a new dimension of misclassification by targeting classifiers across resolution scales. Furthermore, our method inherently avoids overfitting to specific architectures or tasks. While adversarial examples are prone to exploiting inductive biases of particular models or visual tasks, resolution attacks leverage generative capabilities to generalize naturally across various architectures and tasks. We conducted experiments on various CNN architectures, including different frameworks (ViTs, FPN), and across a range of tasks (VQA, image captioning, zero-shot classification). We have incorporated these experiments in the Appendix E. The experimental results underscore the broad applicability of our attack.

Usefulness and Applicability: The dual-semantic capability and source-guided design of RA/RAS enhance its practical relevance in auditing generative models, evaluating robustness, and detecting vulnerabilities in classifiers. Structural consistency with the source image is quantified using the SSIM (Structural Similarity Index) metric, demonstrating that the generated images remain contextually plausible.

By addressing these differences and demonstrating the unique value of our approach, we emphasize that RA and RAS offer significant advancements in adversarial attack methodologies, paving the way for novel research and applications.

Weakness 3: The module proposed lacks novelty, as the separation of control between modules for detail and shape generation is quite standard.

Thanks for this comment. We would like to elaborate on the novelty of our framework:

The core innovation of our work lies in proposing a completely new type of attack—Resolution Attacks (RA)—which systematically exploits the resolution discrepancy between high-resolution and low-resolution image classifiers. This attack concept is novel and unprecedented in the literature.

To operationalize this, we designed a dual-stream denoising network, which ensures the precise alignment of dual-semantic representations at high resolution (PH) and low resolution (PL). This network is tailored to the specific requirements of RA, offering fine-grained semantic control to simultaneously generate distinct, yet semantically aligned outputs for PH and PL.

Additionally, when using the controlnet module in the RAS, our framework further extends flexibility by using different control conditions to adapt to the semantic gap between PH and PL. This ensures more targeted generation of dual-representation images, significantly improving the practicality.

Weakness 4: Key implementation details are missing, such as the resolutions at which images successfully achieve adversarial effects.

We understand your concern about implementation details. About "the resolutions at which images successfully achieve adversarial effects", we have mentioned in the paper. This information is provided in the original submission (Appendix A), where we specify a downsampling factor of 3, resulting in low-resolution images of size 64×64.

We hope the above response can fully address your queries and demonstrate the scientific rigor and research potential of our work.

Thank you for your valuable feedback. Below, we will respond to each of your comments in detail, addressing specific issues while ensuring our reasoning is clear and fully responsive to your concerns.

Weakness 1: The attack scenario presented is quite limited. The paper uses a high resolution of 512×512, but the size of low-resolution images is unclear. Figures suggest it might be 32×32. In practice, this resolution is only common in classifiers trained on small-scale datasets like MNIST or CIFAR-10/100, whereas real-world classifiers, such as those trained on ImageNet, typically use higher input resolutions, making this type of attack less applicable.

Thank you for this comment. We respectfully disagree with the reviewer on these comments that the practicity of this research is limited. We first want to clarify that we leverage low-resolution images of 64x64 (not 32x32) by downscaling the high resolution images with a factor of 3, as we have stated in the original submission (Appendix A). Besides, we further want to clarify that the classifiers we utilize is for 224x224 images, not for 32x32 images. We do this to simulate the real-world scenarios in which classifiers encounter reduced-resolution inputs.

We further note that our proposed resolution attack is highly applicable and relevant in broader and practical settings due to the prevalence of low resolution images in various domains. Specifically, we identify several application scenarios where low-resolution images are commonly used:

• Compressed Internet Images: Images uploaded to social media or online platforms often undergo heavy compression, reducing their resolution significantly. This creates opportunities for resolution attacks to exploit classifiers processing such images.

• Autonomous Driving Systems: In autonomous vehicles, small and distant objects (e.g., traffic signs or pedestrians) often appear as low-resolution entities. Resolution attacks could raise fresh safety concerns in these systems.

• Surveillance Systems: Surveillance cameras often prioritize storage efficiency by capturing video footage at lower resolutions. Resolution attacks could compromise object detection systems designed for such footage.

• IoT Devices: Many IoT devices, such as smart home cameras or wearable devices, operate with constrained hardware and capture images at reduced resolutions. These systems are particularly susceptible to resolution-based vulnerabilities.

Given this, we believe our proposed resolution attack is highly applicable and will influence a wide range of field.

Feedback1.2: As for the application scenario you mentioned, it is more practical for the generated adversarial images to maintain clear visual similarity to the original, clean images. For this reason, I would prefer an approach based on additive noise. Does your method support this additive approach?

Feedback2: The primary concern here is that the generated images, even those in Figure 2(b), lack visual similarity to the source images. Additionally, unlike adversarial attacks, there is no clearly defined bound for the perturbations.

Thank you for your comments. Your main focus is on the issues of the "visual similarity to the source images" and "bound for the perturbations". Since both Q1.2 and Q2 share a common focus, we address them together below.

Our approach aims to assess the semantic robustness of classifiers at different resolutions by generating images using the capability of generative models to create images with dual semantics. Maintaining strict visual similarity to the source images is not the primary goal of our method and does not serve as an appropriate criterion for assessing its effectiveness. Instead, the focus of our approach is to explore a broader search space beyond pixel-level perturbations, enabling more comprehensive evaluations of classifier vulnerabilities across diverse models and tasks.

Methodological Perspective: Semantic-Level Generation vs. Pixel-Level Perturbation

We emphasize that our method differs fundamentally from traditional adversarial attacks. Conventional adversarial attacks typically introduce imperceptible pixel-level perturbations to the original image, carefully computed to exploit the classifier's decision boundaries. These perturbations often have a well-defined bound (perturbation bound), which plays a critical role in assessing the classifier's robustness to adversarial noise.

However, modern classifiers are increasingly exposed to a variety of generative model outputs, including high-resolution synthesized images. Perturbation bounds, while relevant for pixel-level perturbations, constrain the search space of adversarial samples and may not provide a comprehensive evaluation of classifier robustness.

Our resolution attack operates at the semantic level, focusing on generating images that leverage the generative model's priors and semantic representations. This enables the method to explore a broader search space beyond the constraints of pixel-level perturbation bounds. Moreover, our method is not tailored to any specific classifier, making it more versatile for evaluating robustness across different models and tasks.

Visual Similarity and Semantic Gap

Regarding your concerns about the visual similarity of the generated images to the source images, it is important to note that the visual similarity depends on the semantic gap between the high-resolution prompt and the low-resolution prompt. When the semantic gap is small, as demonstrated in Figures 17 and 18 of the paper, the generated images exhibit a much closer resemblance to the source images.

While additive noise-based approaches are effective for preserving high visual fidelity, they inherently remain limited to the pixel level. In contrast, our approach leverages the power of generative models to synthesize images that align with the semantic attributes of the prompts, enabling a novel perspective for evaluating classifier robustness.

We hope the above explanation addresses your concerns and we are open to answering any further questions you may have.

Feedback1.1: Regarding this point, the primary concern is that the image is downsampled from 512 * 512 to 64 * 64, while the classifier expects an input of 224 * 224. It’s unclear whether the downsampled images are directly fed into the classifier or upsampled back to 224 * 224. This aspect needs clarification and justification. In addition, it is possible to conduct some experiments on classifiers with default 64 * 64 inputs.

Thank you for the feedback. Regarding the concern about the downsampling process, we confirm that the downsampled images (64×64 resolution) are directly fed into the classifier. The classifier preprocesses these images using PyTorch's default pipeline, as described in its documentation:

"Accepts PIL Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are resized to resize_size=[232] using interpolation=InterpolationMode.BILINEAR, followed by a central crop of crop_size=[224]. Finally, the values are first rescaled to [0.0, 1.0] and then normalized using mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225]."

Furthermore, in response to your inquiry regarding the "classifiers with default 64 * 64 inputs," as there is currently no classifier specifically tailored for 64×64 resolution inputs, we conducted supplementary experiments using a model trained on the CIFAR-10 dataset (32 * 32) [1]. The experimental results are presented in the table below.

The results demonstrate the effectiveness of our approach even for low-resolution classifiers.

[1] https://github.com/chenyaofo/pytorch-cifar-models/tree/master/pytorch_cifar_models

Thank you for taking the time to review our paper, and for your helpful comments. Below we will address your concerns point by point:

Weakness1 & Question 1: The classifiers tested in the article are primarily based on CNN methods and do not include evaluations of the latest classifiers. I am curious about how transformer architecture models perform under the "resolution attack," and whether methods using feature pyramids could better overcome the "resolution attack." The authors could provide more analysis in this area.

Thanks for the valuable question. We have conducted our proposed Resolution Attacks on Transformer-based models (ViT-B and ViT-L) and Feature Pyramid Network (FPN) methods (Faster RCNN and Mask RCNN). Besides, according to suggestion provided by Reviewer 6DtR, we have evaluated the effectiveness on various vision-language models, including image captioning models (BLIP), Visual Question Answering models (LLAVA) and zero-shot classification models (CLIP). The results are shown in the following tables:

 Additional quantitative results of the Resolution Attack

 Additional quantitative results of the Resolution Attack with Source image

The results further demonstrate that our proposed RA can also successfully attack ViTs and FPN methods, exhibiting robustness across various architecture. Besides, our method can well generalized to various models across different vision tasks. This is not surprising as our method is model-agnostic and focus on the semantic of the image. We have incorporated these experiments in the Appendix E.

Weakness2: Although the authors validate the effectiveness of their method on existing classifiers, they do not provide a detailed comparison with other attacks (such as adversarial attacks). The authors might consider analyzing how the proposed "resolution attack" compares to other generative attacks that also utilize the diffusion model.

Thanks for your advice. We have no doubt that the reviewer understands the fundamental difference in our proposed Resolution Attack (RA) and other attacks. We provide the detailed comparison here and also supplement these comparison in Appendix D.

We would like to emphasize that the Resolution Attack is fundamentally different from other attacks such as traditional adversarial attacks and other generative attacks utilizing diffusion models:

First, RA shares very different goals from other attacks. The Resolution Attack aims to deceive both classifier and human vision while other attacks are mainly focused on machine learning model. Due to this, our method enjoys a natural merit to generalize across different architecture and different vision tasks while other attacks may be easily overfitted to inductive bias of a specific model or vision task.

Second, RA shares very different technical roadmaps from other attacks. RA aims to synthesize dual-semantic images and focus more on the content of the images. Thus, RA requires to search a broader landscape of the image manifold and heavily relied on the semantic-level understanding of the images. (We achieve this by utilizing the strong generative priors of the large-scaled Stable Diffusion.) However, other attacks only involve pixel-level perturbations and search a p-ball around the target image.

Furthermore, the primary objective of RA is to generate images with distinct semantics across different resolutions. This differs fundamentally from traditional adversarial attacks and generative attacks based on diffusion models. As the first work to propose the concept of resolution attacks, we establish a baseline for this novel direction.

Weakness3: The paper touches on applications like face swapping and camouflage without addressing the potential ethical implications.

Thank you for highlighting the need to address ethical considerations. We have now explicitly discussed these aspects in of the revised paper (Appendix G). Specifically, we emphasize that our goal is to explore vulnerabilities in current vision models. Though our approach is under the risk of malicious purpose, our generated image can be employed to fine-tune existing classifiers, further improving their robustness. Besides, the generated images can also be detected by deepfake detectors. In summary, our study seeks to advance understanding in model robustness while advocating for responsible and ethical use of such methodologies in the broader research community.

Thanks for the author's response. Most of my concerns have been addressed, so I will increase my rating to 8.

We are delighted to have addressed your concerns and appreciate your constructive insights, which have significantly enhanced the quality of our paper. Specifically, your recommendation to analyze the performance of Transformer-based models under the resolution attack and to explore methods utilizing feature pyramids has helped us better demonstrate the broader applicability and robustness of our approach. Additionally, your other suggestions greatly contributed to enriching the depth of our work.

We deeply appreciate your feedback and the time you took to review our paper. Thank you once again for your support and for raising the rating of our work.