BackBench: Are Vision Language Models Resilient to Object-to-Background Context?

Evaluating Recent Vision Models. As recommended, we have conducted evaluations on recent models such as DeiT[7] and ConvNeXt[8] in the Table below (also added in Section A.13 in the revised Appendix). We observe a consistent trend of decrease in performance to background changes across these models as well. We observe that DeiT-based models which distill the knowledge from a strong CNN-based model during training show overall improved performance across our object-to-background changes compared to ViT-based models (please see Table 1 in main paper).

Comparison of recent CNN and transformer-based models (Top-1%) against our proposed object-background changes our IN-Nat and IN-Adv datasets. The table is also added in revised Appendix A.13 (Table 14).

Furthermore, we also provide results on DINOv2-based models [9]. Our preliminary experiments with the DINOv2 models, as presented in Table below (see A.14 in revised Appendix), show that DINOv2 models which utilize registers (learnable tokens) [10] during training for learning more interpretable attention maps, provide more robustness to background changes, with significant improvement seen in the adversarial background changes.

Evaluation of Dinov2-based models on IN-Nat and IN-Adv dataset.  DINOv2 model with registers shows higher robustness to background changes, particularly in the presence of color, texture, and adversarial backgrounds. The table is also added in revised Appendix A.14 (Table 15).

We thank the reviewer for the insightful comments. Please find our responses to specific queries below.

On using Existing Foundational Models. Given the large-scale multi-modal training of recent vision-language models and their generalization capabilities, there is a need for evaluation benchmarks going beyond simple common corruptions [1] or adversarial perturbations [2] to better understand their behavior in the wild. The main motivation of our proposed work is to study how changes in the object background can affect the performance of various uni/multi-modal models. To this end, the proposed framework utilizes existing foundational models such as, SAM [3] and BLIP-2 [4] for visual and textual conditioning. Recent relevant works also utilize large-scale pretrained models but for global perturbations, while ours specifically studies the background changes impact on models’ performance. For instance, LANCE [5] (NeurIPS’23) employs the use of pretrained Large Language Models alongside an already proposed image editing technique [6] to induce global modifications in the real images. Such works underscore a shared trend in the field: the creative re-purposing of existing foundational models to forge new ways to evaluate the robustness of uni/multi-modal models. Different from LANCE, our contribution lies in how we have utilized the complementary strengths of different foundational models for our proposed object-to-background benchmarking. Our work preserves the semantics, shape, angle, and color of the original object while changing its composition to the background in the real images.  By combining the capabilities of current image-to-text, image-to-segment and text-to-image-models, we have created a framework that is significantly more adept at handling complex background changes. Furthermore, we use our framework to generate adversarial background changes by optimizing the visual and textual latents of the diffusion model. Our rigorous set of experiments on vision tasks such as, image classification, object detection and segmentation are designed to validate the effectiveness of our framework across uni/multi-modal models.

Generating Diverse Backgrounds. Kindly note that in Figure 3 (main paper), we generate diverse, realistic background changes by a variety of prompts using ChatGPT rather than random seeds. We provide these ChatGPT prompts in Table 7 (revised Appendix A.3). This demonstrates the effectiveness of our text prompt-based diversity in image generation. Additionally, we provide further qualitative results of this capability in the appendix, specifically in Figures 8 and 9 (see revised Appendix A.3), where we display a range of results achieved using these varied text prompts. 

We show the effect of random seeds in Figure 7 in Appendix A.3, where we use BLIP-2 captions for the original image as a text prompt. We fixed this text prompt and generated images with different random seeds. We observe from rows 2, 3 and 5 of Figure 7, that altering the seed while keeping the same caption does not yield significant background diversity across the generated samples. Furthermore, using random seeds only does not provide the fine-grain control needed to benchmark vision-based models, which is our main objective. These observations highlight the importance of our designed textual prompts to generate diverse object-to-background changes for benchmarking current vision-based models.

On Studying Robustness of Vision-Language Models.  In this work, our focus has been on examining vision language models through various lenses, including classification, detection, segmentation, and image captioning. Specifically, we have conducted evaluations of vision-language models like CLIP and EVA-CLIP for zero-shot classification tasks (Table 2 in the main paper, Table 10 in the revised Appendix). For the case of segmentation, our attention has been on assessing the performance of the multi-modal segmentation model SAM (Table 8 in the revised Appendix A.6). Moreover, for the task of image captioning, we have evaluated the robustness of the image-to-text BLIP-2 model (Table 4 in the main paper). Along with these, we study different uni-modal CNNs and vision transformers trained naturally and adversarially. As suggested, we will modify the title to avoid any confusion such as "Backbench:  Are Vision Models Resilient to Object-to-Background Context?"

Thank the authors for the feedback and new results.

I still have concerns regarding the contributions of this work. The data generation approach are not new. Even with ChatGPT prompts, generating the diverse backgrounds with textual conditions has been explored before.

Thanks again for the comments but I'm not fully convinced to raise my rating.

b) Qualitative comparison: Since LANCE relies on global-level image editing, thus tends to alter the object semantics and distort the original object shape in contrast to our approach which naturally preserves the original object and alters the object-to-background composition only. This can be observed in qualitative examples provided in Figures 25 and 26 in revised Appendix section A.9. We further validate this effect by masking the original images and LANCE-generated counterfactual images. We observe an accuracy drop from 97.71% to 84.35% across different models. Note that this is the first step of our approach, where the background is just masked but not optimized for any background changes. However, when the background is masked in LANCE-generated counterfactual images, overall accuracy drops from 97.71% to 71.57%. This shows that the LANCE framework has changed the original object during optimization which is also reflected in Figures 25 and 26 in revised Appendix Section A.9. Therefore, our proposed approach allows us to study the correlation of object to background changes without distorting the original object.

Quantitative evaluation (Top-1%) of LANCE on IN-Adv shows its tendency to alter the original object in contrast to our approach which preserves the original object during optimization. The table is also added in revised Appendix A.9 (Table 13).

References

[1] Darcet et al. "Vision Transformers Need Registers." arXiv preprint arXiv:2309.16588 (2023).

[2] Sitawarin et al. "Part-based models improve adversarial robustness.", ICLR, 2023.

[3] Yuan et al. "CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models." arXiv preprint arXiv:2310.19784 (2023).

[4] Kawar et al. "Imagic: Text-based real image editing with diffusion models.", CVPR, 2023.

[5] Hendrycs et al. "Benchmarking Neural Network Robustness to Common Corruptions and Perturbations", ICLR 2019.

[6] Madry et al. "Towards deep learning models resistant to adversarial attacks", ICLR 2018.

[7] Kirillov et al. "Segment anything.", ICCV, 2023.

[8] Li et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models." arXiv preprint arXiv:2301.12597 (2023).

[9] Prabhu et.al "LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images.", NeurIPS, 2023.

[10] Hertz et.al "Prompt-to-prompt image editing with cross attention control", ICLR, 2022.

[11] Mokady et al. "Null-text inversion for editing real images using guided diffusion models", CVPR, 2023.

Comparison with recent State-of-the-art Method.  We compare with the recently proposed LANCE [9] (NeurIPS’23), which is relevant to our approach for benchmarking robustness. Their work leverages the capabilities of large language models to create textual prompts, facilitating diverse image alterations using the prompt-to-prompt method [10] and null-text inversion [11] for real image editing. We observe the following noticeable difference with LANCE as compared to our proposed approach:

1. LANCE relies on prompt-to-prompt method [10] and null-text inversion [11] for real image editing which allows modification of only specific words in the textual prompt. This limitation can restrict the range of possible image transformations. In contrast, our method uses visual conditioning to better preserve the object semantics, allowing us the flexibility to use textual guidance to introduce diverse background changes.
2. LANCE makes global changes to a given input image which can alter object semantics, e.g., shape distortion (See Table 13 and Figures 25 and 26 in the revised Appendix A.9).
3. Furthermore, LANCE needs adjustments in attention-based manipulation hyperparameters, and the selection of optimal images based on CLIP scores for each sample which is not scalable to larger datasets. In contrast, our approach allows for the evaluation of vision-based models across a much broader range of vision tasks. On the task of classification, we provide an evaluation of 5.5k images across 582 classes of ImageNet (See Figure 32 in the revised Appendix A.12), as opposed to the 15 classes evaluated in LANCE. 

We use open-sourced code from LANCE to compare it against our approach both quantitatively and qualitatively. We use a subset of 1000 images, named IN-ADV, for comparison.  a) Quantitative comparison: We observe that our natural object-to-background changes including color and texture perform favorably against LANCE, while our adversarial object-to-background changes perform significantly better as shown in the table below.

Quantitative comparison (Top-1%) of our approach with LANCE. Our approach performs favorably well for benchmarking the robustness of ViTs and CNNs. The table is also added in the revised Appendix A.9 (Table 12).

We thank the reviewer for the insightful comments. Please find our responses to specific queries below.

Future Directions. Our research work is one of the preliminary efforts in utilizing diffusion models to study the object-to-background context in vision-based systems. Based on our observations and analysis, the following are the interesting future directions.

1. Since large capacity models in general show better robustness to object-to-background compositions, coming up with new approaches to effectively distill knowledge from these large models could improve how small models cope with background changes. This can improve resilience in small models that can be deployed in edge devices.
2. Another direction is to set up object-to-background priors during adversarial training to expand robustness beyond just adversarial changes. To some extent, successful examples are recent works [1, 2], where models are trained to discern the salient features in the image foreground. This leads to better robustness. Kindly see our detailed robustness analysis on DINO-v2 with registers [1] in our response to reviewer-SPdM "Evaluating Recent Vision Models". This shows that similar techniques can improve adversarial training as well.
3. Our work can be extended to videos where preserving the semantics of the objects across the frames while introducing changes to the background temporally will help understand the robustness of video models.
4. Additionally, the capabilities of diffusion models can be explored to craft complex changes in the object of interest while preserving the semantic integrity. For instance, in [3], diffusion models are employed to generate multiple viewpoints of the same object. Additionally, in [4], non-rigid motions of objects are created while preserving their semantics. By incorporating these with our approach, we can study how vision models maintain semantic consistency in dynamic scenarios.

On the Novelty of Proposed Architecture. Given the large-scale multi-modal training of recent vision-language models and their generalization capabilities, there is a need for evaluation benchmarks going beyond simple common corruptions [5] or adversarial perturbations [6] to better understand their behavior in the wild. The main motivation of our proposed work is to study how changes in the object background can affect the performance of various uni/multi-modal models. To this end, the proposed framework utilizes existing foundational models such as, SAM [7] and BLIP-2 [8] for visual and textual conditioning. Recent relevant works also utilize large-scale pretrained models but for global perturbations, while ours specifically studies the background changes impact on models’ performance. For instance, LANCE [9] (NeurIPS’23) employs the use of pretrained Large Language Models alongside an already proposed image editing technique [10] to induce global modifications in the real images. Such works underscore a shared trend in the field: the creative re-purposing of existing foundational models to forge new ways to evaluate the robustness of uni/multi-modal models. Different from LANCE, our contribution lies in how we have utilized the complementary strengths of different foundational models for our proposed object-to-background benchmarking. Our work preserves the semantics, shape, angle, and color of the original object while changing its composition to the background in the real images.  By combining the capabilities of current image-to-text, image-to-segment and text-to-image-models, we have created a framework that is significantly more adept at handling complex background changes. Furthermore, we use our framework to generate adversarial background changes by optimizing the visual and textual latents of the diffusion model. Our rigorous set of experiments on vision tasks such as, image classification, object detection and segmentation are designed to validate the effectiveness of our framework across uni/multi-modal models.

We thank the reviewer for the insightful comments. Please find our responses to specific queries below.

Potential External Factors and Experimental Results. Kindly note that when composing object-to-background change with texture, color, or adversarial patterns, the target models can perceive those as some other class if that pattern or composition is dominant in that class during the training of the models. We discuss the potential external factors and how our proposed approach minimizes the effect of those external factors during benchmarking.

• Possibility of extra objects in the Background: Kindly note that a) we use a pretrained diffusion model that is conditioned on a pretrained CLIP text encoder, this means that the generated output follows the latent space of the CLIP text encoder which is aligned with CLIP visual encoder. Therefore, we can measure the faithfulness of the generated sample w.r.t the textual prompt used to generate it. We can measure this by encoding the generated output and its corresponding text prompt within CLIP latent space. For a given sample, CLIP or EVA-CLIP performs zero-shot evaluation by measuring the similarity between embedding of class templates (e.g. 1000 templates of ImageNet class) with a given image. Thus, if we simply add the template for a textual prompt used to generate the object-to-background changes, then we can measure its alignment with the background changes. For instance, instead of using a “a photo of a {fish}” template for zero-shot classification, we add the relevant template that is with background change, such as “a photo of a {fish} in the vivid colorful background”. In other words, the relevant template represents the object and background change we introduced. We validate this observation on the EVA-CLIP ViT-E/14+, a highly robust model. Using the class templates such as “a photo of a {}”, the model achieves 95.84% accuracy on the original images (IN-Nat dataset), which decreases to 88.33% when our color background changes are applied. However, when using the relevant template, the performance improves to 92.95%, significantly reducing the gap between the performance on the original and color background changes from 7.51% to 2.89%. These results show that accuracy loss from background changes isn't due to unwanted background objects of other classes. Furthermore, we manually assess 2.89% of misclassified samples (very few samples, see Figures 28 and 29 in the revised Appendix A.10). These can be considered the hardest examples in our dataset. We observe that even in such hard cases the model's confusion often stemmed from the complex background patterns instead of the addition of unwanted objects. We observe a similar trend in the case of adversarial patterns as well (see Figure 30 in the revised Appendix A.10). b) Another empirical evidence of how our generated output closely follows the given textual prompts can be observed with BLIP-2 Caption of the original image. In this case, object-background change has similar results as compared to original images across different vision models (Table 2 in the main paper).

• Extension of Objects:  As already detailed in Section A.11 and illustrated in Figure 31 of the revised Appendix, we encountered challenges when dealing with objects that occupy a small region in the image, sometimes leading to certain unwanted extensions to objects. To mitigate this, we filtered our dataset to focus on images where the object covers a significant area. Additionally, we slightly expand object masks computed using SAM to better define boundaries and prevent object shape distortion in the background. 

• Preserving Object Semantics: We preserve object semantics by using strong visual guidance via SAM for precise object delineation. In contrast, recent methods like LANCE [1] make global changes and inadvertently affect the object's semantics. Please refer to the revised Appendix A.9 (Figures 25 and 26) for a qualitative comparison. We also kindly refer to our response to reviewer-ATH5 "Comparison with recent State-of-the-art Method."

The design choices discussed above, such as high-quality data filtering, strong visual guidance, and class-agnostic textual guidance, contribute to the well-calibrated results of our study. This indicates that our results using the conventional metrics such as classification accuracy are well calibrated as well in the context of our high quality of generated data as mentioned above. We note that these choices ensure that the models are primarily challenged by diverse changes in the background, rather than being misled by the presence of unwanted objects. This careful approach underlines the reliability of our findings and highlights the specific factors influencing model performance. 

Our benchmark datasets will be made publicly available, to enable further exploration and build upon our work.

References

[1] Prabhu et.al "LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images.", NeurIPS, 2023.

We use open-sourced code from LANCE to compare it against our approach both quantitatively and qualitatively. We use a subset of 1000 images, named IN-ADV, for comparison. a) Quantitative comparison: We observe that our natural object-to-background changes including color and texture perform favorably against LANCE, while our adversarial object-to-background changes perform significantly better as shown in the table below.

Quantitative comparison (Top-1%) of our approach with LANCE. Our approach performs favorably well for benchmarking the robustness of ViTs and CNNs. The table is also added in the revised Appendix A.9 (Table 12).

b) Qualitative comparison: Since LANCE relies on global-level image editing, thus tends to alter the object semantics and distort the original object shape in contrast to our approach which naturally preserves the original object and alters the object-to-background composition only. This can be observed in qualitative examples provided in Figures 25 and 26 in revised Appendix section A.9. We further validate this effect by masking the original images and LANCE-generated counterfactual images. We observe an accuracy drop from 97.71% to 84.35% across different models. Note that this is the first step of our approach, where the background is just masked but not optimized for any background changes. However, when the background is masked in LANCE-generated counterfactual images, overall accuracy drops from 97.71% to 71.57%. This shows that the LANCE framework has changed the original object during optimization which is also reflected in Figures 25 and 26 in revised Appendix Section A.9. Therefore, our proposed approach allows us to study the correlation of object to background changes without distorting the original object.

Quantitative evaluation (Top-1%) of LANCE on IN-Adv shows its tendency to alter the original object in contrast to our approach which preserves the original object during optimization. The table is also added in revised Appendix A.9 (Table 13).

On using SAM. Kindly note that we use SAM [1] as our image-to-segment model for visual conditioning to preserve the object semantics. This aligns with our objective to study the robustness of vision-based models to object-to-background compositional changes. The requirement to preserve the object semantics needs an instance segmentation model like SAM, which if not used would result in editing the actual object (a limitation other global methods like LANCE suffer from).

References

[1] Kirillov et al. "Segment anything.", ICCV, 2023.

[2] Li et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models." arXiv preprint arXiv:2301.12597 (2023).

[3] Prabhu et.al "LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images.", NeurIPS, 2023.

[4] Hertz et.al "Prompt-to-prompt image editing with cross attention control", ICLR, 2022.

[5] Mokady et al. "Null-text inversion for editing real images using guided diffusion models", CVPR, 2023.

We thank the reviewer for the insightful comments. Please find our responses to specific queries below.

On Novelty. The main motivation of our proposed work is to study how changes in the object background can affect the performance of various uni/multi-modal models. To this end, the proposed framework utilizes existing foundational models such as, SAM [1] and BLIP-2 [2] for visual and textual conditioning. Recent relevant works also utilize large-scale pretrained models but for global perturbations, while ours specifically studies the background changes impact on models’ performance. For instance, LANCE [3] (NeurIPS’23) employs the use of pretrained Large Language Models alongside an already proposed image editing technique [4] to induce global modifications in the real images. Such works underscore a shared trend in the field: the creative re-purposing of existing foundational models to forge new ways to evaluate the robustness of uni/multi-modal models.

Different from LANCE, our contribution lies in how we have utilized the complementary strengths of different foundational models for our proposed object-to-background benchmarking. Our work preserves the semantics, shape, angle, and color of the original object while changing its composition to the background in the real images. By combining the capabilities of current image-to-text, image-to-segment and text-to-image-models, we have created a framework that is significantly more adept at handling complex background changes. Furthermore, we use our framework to generate adversarial background changes by optimizing the visual and textual latents of the diffusion model. Our rigorous set of experiments on vision tasks such as, image classification, object detection and segmentation are designed to validate the effectiveness of our framework across uni/multi-modal models.

Comparison with Recent State-of-the-art Method. As recommended, we compare with the recently proposed LANCE [3] (NeurIPS’23), which is relevant to our approach for benchmarking robustness. Their work leverages the capabilities of large language models to create textual prompts, facilitating diverse image alterations using the prompt-to-prompt method [4] and null-text inversion [5] for real image editing. We observe the following noticeable difference with LANCE as compared to our proposed approach:

1. LANCE relies on prompt-to-prompt method [4] and null-text inversion [5] for real image editing which allows modification of only specific words in the textual prompt. This limitation can restrict the range of possible image transformations. In contrast, our method uses visual conditioning to better preserve the object semantics, allowing us the flexibility to use textual guidance to introduce diverse background changes.
2. LANCE makes global changes to a given input image which can alter object semantics, e.g., shape distortion (See Table 13 and Figures 25 and 26 in the revised Appendix A.9).
3. Furthermore, LANCE needs adjustments in attention-based manipulation hyperparameters, and the selection of optimal images based on CLIP scores for each sample which is not scalable to larger datasets. In contrast, our approach allows for the evaluation of vision-based models across a much broader range of vision tasks. On the task of classification, we provide an evaluation of 5.5k images across 582 classes of ImageNet (See Figure 32 in revised Appendix), as opposed to the 15 classes evaluated in LANCE.