Measuring Dejavu Memorization Efficiently

Thank you for reviewing our paper and raising those important questions!

Weakness 1 - Sample-level granular analysis

In our paper we obtain dataset-level statistics by aggregating sample level results. Appendix C.1 provides examples for Vision and C.2 for Vision language models. We ran additional experiments and shared the results in the 1 page PDF and in the global rebuttal. Please, check global rebuttal section.

Weakness 2 - The reliance on simpler models like Naive Bayes classifiers might limit the method's accuracy in certain complex scenarios

This is a good point. In our scenario for the ImageNet dataset Naive Bayes provided sufficiently good results. Overall, whether it is Naive Bayes or ResNet we observe similar memorization patterns. Our experiments show that many of those observations (for instance lower memorization in pre-trained models compared to the models trained on the subsets of the dataset) are also true for Vision Language Models (VLMs).

Weakness 3 - A risk that the simpler models used for approximating dataset-level correlations might overfit their training data, which could skew the results.

Yes, this can happen. We used early stopping to avoid overfitting. Apart from that, we retrained the models multiple times with random seeds and showed a small variance across five training runs.

Since Naive Bayes is a deterministic classifier, we used a bootstrapping approach to estimate the variance. We randomly sampled half of the examples per class and retrained Naive Bayes classifiers on those subset. We repeated the experiment 5 times to estimate the mean and the variance.

The results are presented in the table below:

The table above represents the error bars for the 2a plot in our paper. Due to limited space in 1-pager rebuttal PDF we represent the results in a table.

Weakness 4 - Biases in the reference models

Yes, this can be the case, however, we have similar observations for dataset, class and sample-level memorization across different types of correlation models (Naive Bayes and ResNet) and different types of representation learning models such as Vision Language Models and Vision Representation models. Our goal in this paper is to show that we can effectively replace two-model déjà vu memorization test with one model. We leave detailed analysis of the biases in the reference models for future work.

Thank you for the detailed review of our paper and the questions!

Question 1- Clarification on Section 4.1.2

Vision Representation

In our work we are not stating that there is a causal relationship between training size and memorization. We hypothesize that pre-trained models memorize less due to larger training sets. [1] shows in Figure 4 that increasing the size of training data doesn’t always increase the memorization score. This result, however, is reported on smaller subsets of the training data. Retraining the model on larger subsets is computationally expensive and we haven’t run that experiment. Our goal was to show the effectiveness of our approach for pre-trained models and not retraining new models.

VLM

For VLMs, previous work [2] (that requires training 2 models; target and reference models; for their memorization test) did show an experiment with different dataset sizes, where they concluded that larger training set size leads to lower population-level memorization. Our contribution is to show that their method can be applied with just one model, but our conclusion regarding dataset size should remain the same as theirs.

Questions 2 - Advice and stronger conclusion of the work

Vision Representation

• Our key contribution is to purpose effective one-model tests that can be used for pre-trained vision models and are comparable to the two model tests in [1]. We do not rank the models based on the amount of memorized data.
• Nonetheless, we observe that DINO memorizes the least amount of data compared to VICReg and Barlow Twins.
• We plan to release a list of ImageNet examples with large dataset-level correlation scores; a list of examples for which the foreground can be inferred easily based on the background crop. This can help the vision community researcher to better estimate whether their models are memorizing certain examples or whether the models make correct predictions due to strong dataset-level correlations.

VLM

• Our key contribution is a method to evaluate the déjà vu memorization with only one model, by utilizing an out-of-box pre-trained language model as a reference model. With this setup we show that the prior work of [2] can be extended to quantify the memorization of pre-trained models provided we have access to the training set. We show that one such pre-trained model (OpenCLIP model checkpoint pre-trained on YFCC15M) suffers from déjà vu memorization, similar to the models we train from scratch on a 40M subset of Shutterstock dataset.
• We hope our evaluation methodology helps future researchers and ML practitioners to evaluate the memorization in different models and compare which model is safe for deployment. While the mitigation strategies are not the focus of the paper, existing mitigation results from [2] can be extrapolated to our setting as our setup mimics theirs except that we do not need to train a second (reference) model from scratch.

Weakness 1 - smaller classifier not memorizing training data

[1] shows that classifiers memorize significantly less than self-supervised models. Apart from that we ensured that Grounded SAM, a tool that was used to annotate background images for Naive Bayes, did not rely on ImageNet dataset during training.

We'd be happy to make those clarifications in the paper.

[1] Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, and Chuan Guo. “Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning.”, NeurIPS, 2023.

[2] Jayaraman, Bargav, Chuan Guo, and Kamalika Chaudhuri. "D'ej\a Vu Memorization in Vision-Language Models." arXiv preprint arXiv:2402.02103 (2024).

Thank you for the thoughtful comments!

Novelty of the work

Original Déjà Vu memorization [2] work requires training two different models with the same SSL architecture on two disjoint splits of the training data and is computationally expensive. The novelty of our work lies in proposing creative ways of estimating déjà vu memorization for any vision and vision language (VLM) pre-trained model using a generic correlation detection classification model. Our approach is less expensive and can be used for multiple pre-trained SSL architectures and exhibits accuracy that is comparable with two model tests proposed in the original Déjà Vu paper [2].

Grounded-SAM's training set

This is a good point, and this was one of our major considerations in annotation model selection. As our annotation model, we use Recognize Anything (RAM), a component of Grounded-SAM. While our representation learning models are trained on ImageNet, RAM does not use ImageNet during training. Their paper also says that ImageNet has an unusual tag presence, and is hence inappropriate for their training task.

Sample-level memorization

In order to obtain dataset-level memorization, we aggregate sample-level memorization results.

Vision

As described in [2]'s Figure 1, for each background crop we predict the foreground label both with a SSL model (e.g. pre-trained VICReg) and correlation classifier. If SSL predicts the correct class and the correlation classifier fails to predict the correct class then we mark that example as memorized. We then aggregate all memorized examples to obtain a dataset level view.

VLM

For VLM learning, we do the evaluation of sample-level precision and recall gap between objects recovered from target and reference models and report the aggregate population level statistics in the main paper.

We include the sample-level statistics both for Vision and VLM in the1-pager PDF (see Figures 1 and 2) in the attached pdf. See global rebuttal for more details.

We would be happy to include these visualizations in the revision.

Two-Model Dependency

Vision

• one of the models is the target model for which we want to measure the memorization. This can be any pre-trained SSL model.
• the other model is the reference model that is used to detect dataset-level correlations. It is trained once on (image crop, foreground label) pairs. We still have two models but 1) we don’t have to train a target model - we can use a pre-trained model and 2) we train a correlation classification model only once and use it as a reference across all pre-trained SSL Vision models.

VLM

• one of the models is the target model for which we want to measure the memorization. This could also be any pre-trained VLM.
• the other model is a pre-trained language model (such as GTE) which we use to capture correlations. We only need to get the captions embeddings for kNN search once with this model. This can be reused for all the target models.

Pre-trained models

For vision, pre-trained models are the target models which we measure the memorization for. We didn't use a pre-trained reference model for vision since we didn't see any such OSS model appropriate for our task.

For VLM we use a pre-trained language model (GTE) [1] as a reference model in the VLM one-model tests. This model is fairly complex and has strong generalization in terms of language model (LM) capabilities as it has been trained on 788M text pairs crawled from various sources from the internet. While this can be replaced with any off-the-shelf language representation model, we would expect similar memorization gaps. In section 4.2.2, the pre-trained models correspond to the target VLMs that are pre-trained on existing datasets and are readily available as checkpoints for evaluation. Here we use the OpenCLIP's checkpoint pre-trained on YFCC15M dataset. The choice of the pre-trained target model will heavily impact the memorization results as the size and quality of the pre-training data will directly control the memorization capability of the model. For instance, the OpenAI CLIP model pre-trained on their 400M private data corpus will have different memorization than the YFCC15M pre-trained model. However, we would need access to the pre-training set to evaluate the memorization. Larger pre-training data typically leads to smaller population-level memorization.

Limitations of our work

Vision

• The approach might be less successful if the training set for the correlation reference model is too small and the model is not able to learn dataset level correlations effectively. In our experiments we ensured a large subset (300k per class) for ImageNet. We recommend using large and representative sample size for training correlation classifier.
• Although [2] shows that supervised models memorize significantly less compared to the SSL models, it is still possible that ResNet memorizes some of its training data. In contrast Naive Bayes doesn’t memorize its training data, however, it is a much simpler optimizer. Overall we have similar observations based on both types of correlation classifiers.

VLM

• The pre-training data of the reference LM either has an overlap with the target VLM’s training data, or has a superior generalization capability than the VLM. In either of the cases, the reference model might match or outperform the VLM in which case the memorization gap will be underestimated. This could happen if, for instance, we use a superior LM that has better reasoning capability to infer beyond dataset level correlations.
• On the other hand if the reference LM is simplistic or has a poor language understanding, then the memorization gap will be overestimated.

[1] Zehan Li, et.al. "Towards general text embeddings with multi-stage contrastive learning" arXiv, 2023

[2] Casey Meehan et. at. “Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning”, NeurIPS, 2023.

Thank you for the response, the authors have adequately answered my questions, I will raise my score.