Generate Universal Adversarial Perturbations for Few-Shot Learning

We truly appreciate your valuable comments. In the following, we respond to the concerns.

Q1: It is confusing between the meaning of task and class. How can the authors state that the downstream task becomes less similar to the pre-training task with higher ASR in Fig. 3 (a)?

Based on the introduction of FSL(lines 91-101), 'task' means a unique classification problem that the model encounters. 'class' or 'way' denotes the individual categories within those tasks. The pre-training task is a 64-way classification problem. As the downstream tasks transition from 64-way to 5-way, the distance to the pre-training task is increased, and a decrease in the ASR can be observed. The decline in ASR suggests that the UAP's performance degrades when faced with downstream tasks that diverge significantly from its pre-training tasks, which supports our proposition of the task shift.

Q2:Why using GAP in Sec. 4? Why don’t the authors use Base classifier?

In Section 4, our motivation for employing GAP is to compare the conventional UAP generation approach under the scenario of FSL. This section is to examine the challenges for UAP in FSL. The Base classifier is introduced in Section 5 as part of our attack methodology. The sequential presentation is intentional, allowing us first to explore the existing methodologies and their limitations, then emphasizing the need for our novel framework.

Q3: The reviewers cannot understand the difference and relationship between Base Classifier, Proxy Linear, and Proxy Protos. Why the performance of the proposed proxy linear is not as good as base linear?

Thank you for the reminder. Allow us to clarify the distinctions and relationships among the terms.

• Base Classifier: As defined in line 192 of our paper, this refers to the classifier pre-trained on the base dataset. It serves as our starting point before further adaptations.
• Proxy Linear: As mentioned in line 201, the Proxy Linear classifier is trained through proxy tasks. This is a key step to handle the task shift.
• Base Linear: Defined around line 229, this refers to the linear classifier that trained on tasks sampled from the based dataset, which is a further improvement from the proxy linear classifier as it diminishes the semantic gap from proxy dataset to base dataset.
• Proxy Protos: Defined at line 253, this represents our final approach, where we employ prototypes derived from the proxy tasks to enhance the performance.

Q4:Why the attacker cannot access the base dataset?There is no definition of the proxy dataset in the previous works

Regarding the attacker's inability to access the base dataset, we have outlined this threat model in Section 3.1, where we introduce a more challenging scenario(same threat model with [48]). This setup reflects real-world conditions more accurately, as a large amount of data, especially proprietary corporate datasets, are typically not publicly available due to privacy concerns and intellectual property rights.

As for the definition of the proxy dataset, we would like to clarify that it has been introduced in reference [45]. This prior work conducts a comprehensive analysis of the mutual information between images and perturbations, thereby establishing the feasibility of employing a proxy dataset to perform the UAP attack.

Q5: Why did the authors choose the combination of {Cifar, Mini} and {Mini, Tiered}? Why the performance of the Mini proxy dataset is generally higher than the Cifar dataset?

Thank you for your question. As mentioned in lines 165-170, the victim model is trained on MiniImageNet. Consequently, when the proxy dataset coincides with MiniImageNet, it simulates a scenario where the attacker has access to the pre-training dataset, which is advantageous and leads to generally higher performance due to the diminished semantic shift between the proxy and the base dataset.

The choice of different datasets is intentional and serves to illustrate four distinct scenarios. {CIFAR, Mini} represents the cases that the proxy dataset matches (Mini) and doesn't match(CIFAR) the pre-training dataset. {Mini, Tiered} represents the cases that the downstream dataset matches (Mini) and doesn't match(Tiered) the pre-training dataset.

When CIFAR is selected as the proxy and Tiered as the downstream, it embodies the most general and challenging scenario where the attacker suffers both semantic shift from the pre-training to the proxy and also to the downstream dataset. Conversely, using MiniImageNet for both proxy and downstream datasets reflects an idealized condition for the attacker, leading to optimal attack performance.

Q6: The legends in Fig. 6 are confusing.The authors should analyze the performance degradation in Fig.6.

Thank you for the reminder, we have revised the presentation. The analysis is provided in lines 331-339. From fig6, we observe that the ASR remains relatively high when transitioning from 5-way 1-shot to 25-way 1-shot, and from 5-way 1-shot to 5-way 20-shot scenarios(5-1 and 5-5 for downstream tasks). Meanwhile, a noticeable degradation in performance emerges as the ways and shots deviate further from the downstream tasks. This aligns with our task shift claim and suggests that a rough estimation of the downstream tasks is sufficient for an attacker to discern the task bias and produce transferable UAPs.

Q7:What is identical and different part between the proposed method and the [48]?

Thanks for the suggestion. While our threat model aligns with that of [48], we have introduced key advancements in addressing the two shifts:

1. Task Shift: Unlike [48], our method is designed to handle scenarios comprising diverse downstream tasks with limited samples, which [48] doesn't take into consideration.
2. Semantic Shift: [48] employs contrastive learning loss to mitigate semantic shift, our ablation studies (lines 322-329) reveal that our strategy outperforms that using contrastive losses in the FSL scenario.

We truly appreciate your valuable comments. In the following, we respond to the concerns.

Q1: It’s not clear if the task and semantic shifts still hold on larger-scale datasets like meta-dataset.

Thank you for your valuable suggestion. Due to constraints of time and space, we select three distinct sub-datasets from the meta-dataset that significantly differ from miniImageNet and conduct experiments on it. These datasets include CUB, Omniglot, and GTSRB. We utilized these datasets alternately as proxy and downstream datasets to conduct comprehensive tests for our framework's generalizability. The results are shown in table1 and table2 in 'Author Rebuttal', which demonstrate that even when scaled up to larger and more diverse datasets such as those within the meta-dataset, our approach can still achieve SOTA attack efficacy across all three FSL paradigms. This consistency in performance highlights the efficacy of our method in tackling the challenges posed by both semantic and task shifts.

Q2: The paper lacks discussions regarding how the proposed proxy tasks relate to/differ from the existing meta-learning UAP with the similar use of few-shot learning tasks.

Thank you for your valuable comment. We discussed the connections and distinctions between our proposed proxy protos and existing meta-learning-based UAP methods(e.g., LFT).

In terms of similarities, both our work and LFT share the same goal of generating UAPs that can effectively impair model performance on a wide range of unseen test images.

However, several key differences distinguish our approach from LFT and similar meta-learning-based UAP methodologies:

1. Threat Model: A fundamental difference lies in the threat model we adopt. Unlike LFT, which assumes access to both source data and downstream data during the generation of UAPs, our framework is proposed under a more strict attacker scenario. Specifically, our attacker is not only denied access to the source dataset but also has no visibility into the downstream data. This setup reflects the real-world scenarios where attackers may have severely limited information.
2. Attack Scenario: While LFT focuses on generating UAPs tailored to attack standard classification tasks, leveraging few-shot learning techniques as part of its training regime, our method is aimed at a broader spectrum of few-shot tasks.
3. Application of UAPs: Another distinction relates to how the generated UAPs are applied. In the case of LFT, even after utilizing a meta-learning scheme to generate UAPs, these perturbations need to be additionally fine-tuned on the specific downstream task using a small number of examples. Conversely, our methodology employs a generator that yields generalized UAPs capable of being deployed directly without the need for further fine-tuning on the target tasks.

In summary, while both our work and prior studies like LFT explore the generation of UAPs within the context of few-shot learning, our research introduces a stricter threat model, extends the scope to a variety of downstream tasks, and innovates by generating UAPs that do not require task-specific fine-tuning. These points of divergence highlight the novelty and significance of our contribution to the field.

Q3: The paper can benefit from a more comprehensive comparison against recent UAP frameworks (e.g., PAP), with more in-depth discussions.

Thank you for the insightful suggestion. We have incorporated the PAP method under the few-shot learning scenario and made a comparative analysis. Through the experimental results, we delve into how PAP tackles the challenges posed by task and semantic shifts. Due to the space limit, PAP results are demonstrated in table3 and table4 in the reply to the reviewer id8n.

PAP mitigates the impact of fine-tuning by perturbing only the lower-level features. Additionally, it uses learnable Gaussian noise for data augmentation. However, it is crucial to note that PAP achieves generalizability by sacrificing model parameters used in generating UAPs, which comes at the cost of requiring access to the base dataset and a reduction of performance on the base dataset. In contrast, our proposed method demonstrates effectiveness not only on the base dataset but also exhibits superior cross-dataset performance without necessitating access to the base dataset. This feature enables our approach to bridge wider semantic gaps effectively. Moreover, our work uniquely addresses the task shift issue in the few-shot context, a dimension overlooked by PAP.

Q4:Could the improvement be due to the base distribution being more like that of the novel dataset instead of due to enhanced transferability?

We acknowledge your insightful observation. We agree that the improved performance could stem from the base distribution's inherent similarity to the novel dataset's distribution, which potentially leads to better results compared to the proxy distribution. This is the exact explanation for why Base Linear outperforms Proxy Linear.

However, an attacker does not have access to the base distribution in practice. So, during the process of crafting UAPs, the key lies in approximating the base distribution as closely as possible to effectively generalize to the unseen novel distribution. Our approach leverages the generalization capability of the encoder to extend its understanding to the UAP. By doing so, the generalizable UAP can be obtained through various proxy datasets and models.

Q5: What’s the rationale of removing log in eq(3)?

Thanks for the question. We need to mention that the $log()$ function in eq(3) is not the $log()$ inside the Cross-Entropy loss, but an additional component introduced from the GAP method, aimed at boosting performance. We remove it to demonstrate the fundamental performance of our framework, ablating the effect of the additional $log()$. The most basic form of the negative cross-entropy loss is $loss = -\mathcal{H}(f(x+\delta), y)$.

We truly appreciate your valuable comments. In the following, we respond to the concerns.

Q1: All datasets are natural images, and I wonder if the experiments can be improved by trying on out-of-domain datasets, e.g. Omniglot or CUB.

Thanks for your constructive suggestion. In addition to the original three natural image datasets, we have supplemented our study with three cross-domain datasets: CUB, Omniglot, and GTSRB (Traffic Signs), using them alternately as proxy and downstream datasets. Meanwhile, we select Baseline, ANIL, and ProtoNet as representatives of the finetuning, meta, and metric-based FSL paradigms. The experimental results are demonstrated in table1 and table2 in 'Author Rebuttal', which demonstrate that even with a substantial domain gap between datasets, our approach can still achieve SOTA attack efficacy across all three FSL paradigms.

Q2: In Tab 2 and 3, sometimes the ASR is the best when victim and proxy datasets are different. This may need more analysis and ablation.

Thank you for raising this point. We have carefully examined the data and observed that this phenomenon occurs specifically between the miniImageNet and tieredImageNet datasets. We speculate that this is due to the similarity and overlap between the images in the mini and tiered datasets. Consequently, in some cases, we observe higher ASR when the proxy and victim datasets are different.

Q3: More ablations on the algorithms (lines 275 to 278) are needed. For example, is 5-way-1-shot always the best proxy task?

We appreciate your suggestion, and we would like to clarify that we have indeed conducted ablation studies to explore the impact of varying the number of ways and shots in our proxy tasks, as outlined in lines 331 to 339. Aligning with the setup detailed in Section 5.1, we fixed the number of shots at 1 and varied the ways (from 5-way up to 35-way), and conversely, we held the way constant at 5 and increased the shots (from 1-shot up to 30-shot). The downstream tasks remain at 5-way 1-shot and 5-way 5-shot.

The results are depicted in Figure 6, which shows that the ASR for downstream tasks remains relatively high when the proxy tasks are within 25-way and 20-shot. Meanwhile, as both the way and shot numbers deviate further from the actual downstream task, a notable decline in attack effectiveness can be observed. This suggests that a rough estimation of the downstream tasks is sufficient for an attacker to discern the task bias and produce transferable UAPs.

Q4: In Fig 1 and line 142, we mention that GAP cannot generate UAPs for metric-based models. I am curious can we use the base classifier for the attack?

Thank you for the insightful suggestion. Theoretically, the suggestion is feasible. When attacking metric-based models using GAP, since the attacker does not have direct access to the pretraining and downstream datasets, he could train an additional base classifier based on the proxy dataset, which would then be used to generate UAP aimed at the downstream tasks. However, there's a crucial consideration: the classifier trained on this proxy dataset inherently introduces semantic shift, as it is tailored to the distribution of the proxy data rather than the actual target data. Consequently, UAPs generated through such a classifier are likely to exhibit diminished effectiveness.

To provide empirical evidence, we conducted experiments where CIFAR-FS served as the proxy dataset, while mini-ImageNet was employed both for pre-training and as the downstream dataset. We utilized the ProtoNet with ResNet12 as the backbone for our metric-based model. Our results, outlined below, illustrate that when using this approach to generate GAP-based UAPs, the performance in both 5-way-1-shot and 5-way-5-shot downstream tasks is significantly inferior to the methods that directly utilize the original model for UAP generation.

Q5: Which setup is used for reporting numbers in Fig 4(b)?

Figure 4(b) illustrates the performance of our attacking framework, whose setting is detailed in section 5.1(lines 180-193). Within this framework, we choose a model pre-trained on the train split of the mini-ImageNet dataset with a ResNet12 backbone as our victim model. The train split of the CIFAR-FS dataset is chosen as the proxy dataset while the test split of the mini-ImageNet dataset is chosen as the downstream dataset.

Dear Reviewer gmrg,

Sorry to bother you again. Since there are only few hours left in the discussion phase, we wish to know if your concerns have been addressed or if you have any additional concerns. Looking forward to your reply.

Best regards,

Authors

We truly appreciate your valuable comments. In the following, we respond to the concerns.

Q1: Authors attributed the bad transferability of GAP in FSL to the task shift. However, one more relevant issue could be the small number of examples used for generating GAP-based UAP.

Thanks for the insightful comments. We would like to clarify that whether in the traditional scenario with GAP or in the FSL scenario, the generation of UAP is safeguarded by a substantial amount of image data. When generating the UAP using the GAP method in FSL, the attacker leverages a pre-trained model(comprising an encoder and a classifier) to conduct the pre-training classification task (such as 64-way classification) and optimizes the classification loss in reverse to achieve the attack effect. Consequently, all images from the proxy dataset are utilized during the generation of UAP. In the downstream finetuning stage, the model provider fine-tunes the model based on the downstream tasks (training a new classifier with a small set of samples). The attacker cannot manipulate this procedure as he doesn't know what the downstream tasks will be. When the fine-tuning is complete and the user is ready to infer with the new model, the attacker implants the unique UAP onto the images uploaded by the user, thereby executing the attack.

Consequently, we claim that the decline in GAP's effectiveness when applied to FSL is attributed to the task shift and the semantic shift. As these two shifts are progressively mitigated, we achieve a substantial enhancement in attack performance.

Q2: For training of UAP generator, just one 5-way-1-shot task is used or 5-way-1-shot tasks are sampled from a dataset?

Thank you for your question. We sample multiple 5-way 1-shot tasks from a dataset. The fundamental distinction between UAPs(image-agnostic) and image-dependent adversarial perturbations lies in the data required in their generation process. A single UAP is designed as a cumulative effect of perturbations across many individual samples, which enables it to be effective against a wide range of images. This process is crucial for its universal applicability across different inputs. Meanwhile, in the setting of few-shot learning, it is also necessary to first pre-train a transferable base model using a large amount of samples, and then generalize it to new downstream tasks with a limited number of samples.

Q3: Why was GAP chosen as a UAP generator? How does your method compare with recent UAP methods?Why is a comparison with state-of-the-art UAPs not provided?

We chose GAP due to its representativeness as a classical method and its convenience for an extension to the FSL scenario, but our comparisons extend well beyond it in the paper(lines 312-320, table4 and table5). We benchmarked our approach against several methods(including the recent AdvEncoder), across six different few-shot learning paradigms, two distinct downstream tasks (5way-1shot and 5way-5shot), and two victim datasets, outperforming all.

Regarding the similar motivation of the PAP[1] method to ours (both are generalizing UAPs to unseen downstream tasks), we've also evaluated its performance in FSL. PAP perturbs only the low-level feature to mitigate the effect of fine-tuning, which has proven effective in the original paper. However, given the scarcity of classes and samples in few-shot scenarios, PAP underperforms in FSL, The results are shown in table3 and table4.

Table 3: 5-way 1-shot results.

Table 4: 5-way 5-shot results.

Q4: Three FSL datasets used are closely related. How does your method work for a significantly different downstream task?

Thank you for your valuable suggestion. In addition to the three FSL datasets, we have augmented our experiments with three out-of-domain datasets to further validate the generalizability of our method. These include CUB(fine-grained bird images), Omniglot(handwritten characters), and GTSRB(traffic sign images). We utilized these datasets both as proxy datasets and downstream datasets in extended experiments.

The experimental results are summarized in the table1 and table2 in the 'Author Rebuttal', which shows that our method is capable of achieving state-of-the-art adversarial attack performance even when there is a significant domain gap between the datasets.

[1] Pre-trained Adversarial Perturbations.

Dear Reviewer id8n,

Sorry to bother you again. Since there are only few hours left in the discussion phase, we wish to know if your concerns have been addressed or if you have any additional concerns. Looking forward to your reply.

Best regards,

Authors