Private Attribute Inference from Images with Vision-Language Models

We sincerely thank the reviewer for their time and effort spent reviewing our paper. We also appreciate the reviewer's acknowledgment of the studied problem being interesting, our evaluation and presentation. We hope our answers below address the reviewer's concerns.

Q1: Does the accurate, scalable, and automated inference of detailed personal attributes from images posted on pseudonymized platforms pose a privacy concern?

Yes, we strongly believe that the demonstrated inferences amount to a serious privacy threat. First of all, while it is indeed possible for humans to also infer such attributes, human inferences cost around 100x more money and 480x more time (we refer to Q2 and Q3 of Reviewer T2ze for detailed inference costs). As such, using VLMs for this task is a fundamental paradigm change. Notably, VLMs enable these inferences at a large and automated scale, questioning the privacy through obscurity assumptions one usually operates under on such platforms.

Additionally, as detailed in the paper, key privacy regulations consider data that enables the inference of personal information protected data—something not commonly associated with snapshots of parking lots posted in largely non-privacy-concerning contexts.

Further, our view that large-scale personal attribute inferences from online content not intended to be personally identifiable are shared by both the academic community and the public. This is highlighted by similar works [1] on text receiving widespread attention, including an ICLR spotlight and a privacy policy award. Further, similar issues are widely discussed in the community [2,3], as well as real-world applications [4].

All in all, our work is the first to show that personal information is inferable at a large scale from inconspicuous images posted online, which is, in line with other reviewers, a concerning and relevant finding.

Q2: “What are the takeaways of your works in terms of concrete recommendations for society, platform owners, regulators, and privacy researchers?”

Society: We believe that our work can raise awareness among the public that sharing images, even on anonymous platforms where they thought to have taken enough care to obscure their identity, can be revealing of their personal information. We hope that, with awareness of this, users can adjust their online behavior and become more conscious of their privacy.

Platform Owners: Anonymous platforms have to be aware of such risks, and may adjust privacy promises to their users accordingly. Educating their users of the full extent of the risks would be also desirable. At the same time, they could also take a conscious approach to make it harder to process their data at a large scale, obstructing scalable inferences.

Regulators: We believe that in some jurisdictions, such as the EU, there are already strong regulatory protections for private data in place. As such, the important takeaway for regulators is to see how ubiquitous/heterogenous personal data really is.

Privacy Researchers: We believe that for privacy researchers, in the face of these privacy risks posed by foundation models the key challenge to tackle is to develop both model side and user side mitigations.

We thank the reviewer for this insightful question, and we will expand our paper in its next revision with a more detailed discussion.

Q3: Does the used evaluation dataset impact the qualitative conclusions drawn by the paper?

No, we believe that the key message of our paper is qualitative, namely that privacy-infringing inferences on real-world data at scale are possible, and this message is not impacted by the exact quantitative results obtained. Notably, the dataset that we have used for this work is sourced exactly where we see the examined threat: from real-world online forums. As such, any accurate inference on our dataset corresponds to a real-world privacy risk. This is also the reason why we have taken such care in constructing the dataset (labeling in-house) and not releasing it to the public.

Q4: Would an evaluation against humans with OSINT experience be possible?

While such a comparison would certainly be interesting, it is not the focus of the paper (see Q3). Further, as in our case, we already spent 100x more money and 480x more time on labeling than VLMs, a cost difference hard to overcome with domain experts. Finally, we believe that with our current real-world data, such an experiment would be ethically highly problematic, particularly when tasking outside individuals to infer personal data from real-world individuals.

Q5: What is the impact of the sampling temperature parameter on the quantitative inference results?

For this, we run GPT4-V on temperatures 0.0, 0.2, and 0.4, achieving 77.6%, 77.03% (average of three runs: 76.2%, 77.1%, 77.8%), and 77.3% accuracy, respectively. As such, reasonable temperature levels have little impact on the quantitative results. We will include these additional results in the next revision.

Q6: Could you please discuss how inference uncertainty reflects on your findings?

We agree with the reviewer that uncertainty estimates could help in real-world scenarios in case of inaccurate inferences. Qualitatively, similar to [5], we find that models are much less prone to hallucinations when they are tasked with directly reasoning about a given input (compared to, e.g., underspecified QA settings). Actively recognizing this provides an interesting avenue for future research in this area.

References

[1] R Staab et al. Beyond Memorization: Violating Privacy via Inference with Large Language Models. ICLR 2024.

[2] Schneier, B. (2023, Dec). AI and Mass Spying. Schneier on security.

[3] Eggsyntax. (2024, May). Language models model US. LessWrong.

[4] Brewster, T. (2023, Nov) Chatgpt has been turned into a social media surveillance assistant.

[5] R Staab et al : “Large Language Models are Advanced Anonymizers”, 2024; arXiv:2402.13846.

We would like to thank the reviewer for their time and effort spent reviewing our paper, for their insightful and detailed feedback, and for their positive overall assessment of our work. We especially appreciate the reviewer’s acknowledgment of the criticality of the studied privacy issue and for their compliments of the constructed dataset and presentation of the work. Below, we address the reviewer’s questions and comments.

Q1: Could you please elaborate how the insights on the privacy issue discussed in the paper depend on the dataset?

Definitely. While we agree with the reviewer that the exact quantitative results presented in the paper may vary depending on the dataset, we believe that the key message of our paper is qualitative; namely that such privacy infringing inferences from ordinary images posted on anonymous forums are indeed possible at scale. We constructed our dataset in order to verify (or reject) this hypothesis, which we believe to have achieved, especially given the fact that our dataset is comprised of real-world images posted on an anonymous forum (Reddit), and as such, any inference success on our dataset has direct real-world implications. Unfortunately, due to our resource limitations and the expensive labeling procedure, we could not extend the dataset with more samples (we refer to Q2 and Q3 below for more details). Nonetheless, we also believe that constructing large-scale and ideally public benchmarks for investigating and mitigating the privacy infringing inference capabilities for VLMs is interesting and important future work. However, to provide some robustness statistics of our results, we have (1) run our inferences three times at temperature 0.2 obtaining accuracies of 76.2%, 77.1%, 77.8% with GPT4-V, showing little variation, and (2) calculated a CI for our reported result in the paper at temperature 0.0 for GPT4-V: 77.6% +- 3.5% at confidence level 95%. We will include these additional results in the next revision.

Q2: “What is the time spent on filtering the subreddits and on annotating the images separately?”

Filtering the subreddits was based on fixed heuristics and as such it did not take any significant time. Annotating the images required around 1 week of full time work, i.e., roughly 40 hours.

Q3: “What are the estimated financial and time costs of human annotation compared to VLM-based annotation methods?”

As mentioned in Q2, the labeling of the dataset cost around 40 hours of human work. Using the hourly rates set by our institution (USD 35/h) this amounts to USD 1400 of labeling costs. At the same time, using the GPT4-V API, we spent less than USD 12 for inferences on the whole dataset, conducted in around 5 mins with further parallelization possible. As such, GPT4-V inferences are more than 100x cheaper and around 480x faster than human labeling, highlighting the concerning scalability of VLM inferences vs. relying on human annotators for privacy infringing inferences. We thank the reviewer for raising this important point, and will include these numbers in the next revision of our paper.

We thank the reviewer for their time and efforts spent reviewing and for their insightful feedback. We are especially appreciative of the reviewer’s recognition of our attack’s novelty and our extensive experimental evaluation. Below, we address the reviewer’s comments and questions.

Q1: Would it be possible to construct a baseline method relying on classical supervised learning techniques?

Fundamentally, any baseline method relying on supervised learning would require a specialized training dataset on which it has to be trained. As such, mounting a privacy attack using such methods would require significantly stronger assumptions on the adversary. At the same time, and as we argue in our paper, VLMs pose a high privacy risk through inference exactly because of the low barrier of entry for adversaries.

While we agree with the reviewer that such a baseline evaluated directly on our dataset would provide an interesting anchor point for our results, unfortunately, due to the highly diverse and challenging nature of the domain this is not feasible.

To underline this argument, take the following examples of how the VLM inferences have been made: (1) an office room, where among many of the items there is one with a University of Florida logo; (2) the interior of a New York public library; (3) sunset, with a view of a few buildings in Denver, in the foreground an open bottle of German beer; (4) just a picture of an at home office room, stylistic elements indicating the (likely) sex of the author.

Each of the above inferences relies on specific task-dependent knowledge that would need thousands of data points to learn in a supervised setting. As labeling even the few hundred examples in our dataset is considerably expensive (~USD 1400, please see our responses to Q2 and Q3 of Reviewer T2ze for more details), obtaining such a dataset for each possible inference task is not feasible. Clearly, in order to excel at such inferences, the VLMs have to rely on the vast world knowledge obtained during unsupervised pre-training both on image and text data.

Further, the closest direct comparison on a related domain that we can obtain is comparing the performance of VLMs against “traditional” machine and deep learning methods on HAI and PAI datasets, where, as mentioned also in the paper, visual foundation models have shown promising performance over prior methods [1-3]. As such, we also believe that even if one could construct supervised methods for some of the inference types, VLMs would still prevail.

Q2: Could you please expand the discussion on the threat of adversaries replicating your attack and on possible defenses?

Certainly. In general, we share the reviewer’s concern, and it was our main motivation for conducting this study to raise awareness about the inference-based privacy threat VLMs pose such that the community can (1) act against it by mitigating the issue, and (2) people can adjust their online practices in the face of this privacy threat. In line also with our concern, before making a copy of this study available anywhere, we first informed the key entities that produced the examined VLMs about our findings as part of a responsible disclosure. All in all, in line with the common stance in the privacy community, we believe that privacy through obscurity is not robust; which is why we think it is important to raise and systematically evaluate this issue in the form of this study.

Regarding potential mitigations against the attack, as already discussed in Section 6 of the paper, we believe that both internet users and VLM providers can act towards reducing the privacy risks posed by VLMs. From the providers’ side, our findings could serve to strengthen the safety alignment of the models, specifically targeting privacy infringing inferences. From the users’ side, we hope for anonymization tools that could extend to images, by, e.g., removing revealing clues from images using generative modeling.

Nonetheless, most importantly, we believe that the first, and most crucial step, towards responsible and privacy preserving practices of VLM development and use is awareness of the full extent of possible privacy issues these models may pose. Here, our paper focuses on raising and demonstrating the issue of VLMs enabling privacy violating inferences at a large scale.

References

[1] X Cheng et al. A simple visual-textual baseline for pedestrian attribute recognition. IEEE TCSVT 2022.

[2] M Castrillón-Santana et al. Evaluation of a visual question answering architecture for pedestrian attribute recognition. CAIP 2023.

[3] X Wang et al. Pedestrian attribute recognition via clip based prompt vision-language fusion. arXiv 2023.

We thank the reviewer for raising their score and if they have any remaining questions or concerns, we are eager to engage in further discussion until the end of the author-reviewer discussion period.

First, we would like to sincerely thank the reviewer for their time and effort spent reviewing, their insightful comments, and for their highly favorable assessment of our work. We are especially thankful for the reviewer’s acknowledgement of the relevance and timeliness of the presented privacy threat. Below, we address the reviewer’s comments and questions.

Q1: Would it be easy to defend against the presented inference-based privacy attack?

Crucially, independently of how easy it would be to defend against such privacy inferences on the model provider’s side, the fact that these privacy violations are possible with current available models is concerning. Especially, since open-source models are also capable of highly accurate inferences, meaning that even if model providers would make preventative adjustments, these previously uploaded open-source models would still be available to conduct privacy-infringing inferences.

Further, while it is hard to exactly anticipate the difficulty of provider side defenses, we believe that it would be a difficult undertaking to try to prevent privacy infringing inferences while maintaining utility on other tasks. We believe so, as we have observed (as also stated in the paper) that the capability of the models to conduct such inferences is highly correlated with their general capabilities. Additionally, we know that some of the examined models (e.g., GPT4-V) already are supposed to be aligned against such inferences, yet we were able to harness these models without too much effort put into jailbreaking them. In fact, for this rebuttal, we tested our attack on the newer GPT4-o model, which tends to score higher in alignment benchmarks than GPT4-V [1], and achieved even higher accuracy (80.7%); showing that alignment against such inferences is at the moment not strong enough.

Nonetheless, as also argued in the paper, we strongly believe that developing adequate defenses, both on the providers’ side and on the users’ side is crucial going forward in order to enable people to exercise their right to privacy even in the age of VLMs. In this regard, we make the first important step into this direction by pointing out, and systematically evaluating the inference-based privacy risks these VLMs pose; hopefully raising awareness in the broader privacy and machine learning community.

Q2: Could SFT be an effective method to defend against the attack?

We believe that both SFT, and especially preference tuning, such as PPO or DPO could be promising strategies on the model provider’s side to decrease the likelihood of a model answering a privacy infringing inference query. However, conducting such experiments is beyond our means unfortunately as (1) even the examined well-performing open-source models are extremely large to handle for training, and (2) this would require vast amounts of training data that is even more expensive to obtain (please see our responses to Q2 and Q3 of Reviewer T2ze for more details). Nonetheless, we agree with the reviewer and believe that it is an important avenue both for future scientific work and for commercial providers to explore the possibilities of preventing privacy-infringing inferences through finetuning or preference tuning. As part of our responsible disclosure we therefore directly contacted all LLM providers used during this study.

References

[1] Z Ying et al. Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks. arXiv 2024.