SEEKER: Semi-Supervised Knowledge Transfer for Query-Efficient Model Extraction

2. Explain the motivation behind each part of the framework.

We appreciate the suggestion of the reviewer. Here, we would like to introduce how each component of the method contributes to the overall performance and how they are incorporated into the framework.

Our framework comprises four main procedures: offline semantic alignment, online semantic alignment, query generation, and supervised training. The offline semantic alignment belongs to an independent offline pre-training stage, and the adversary iterates through the online semantic alignment, query generation, and supervised training procedures during the online stage. Among these procedures, conventional model extraction attacks only include query generation and supervised training procedures in the online stage, i.e., generate query inputs, query the victim model, and use the victim feedback to optimize the query generation process. Our framework improves the existing pipeline by 1) introducing an offline stage (offline semantic alignment), 2) designing the online semantic alignment to assist the conventional supervised training scheme, and 3) proposing a novel aggregation generator for the query generation process.

Offline semantic alignment. In contrast to existing online attacks, we design an offline stage to pre-train the substitute model. During this stage, the adversary can not directly interact with the victim, but have access to a public dataset. Under such circumstances, we design an offline semantic alignment strategy that leverages the public dataset to optimize the encoding layers of the substitute model in a self-supervised manner. Notably, we identify semantic consistency as a common property in neural networks and utilize the prior to train the substitute model. We also design a symmetric network architecture and projection layers that are suitable for the offline stage.

Online semantic alignment. In addition to the offline semantic alignment, we further develop a self-supervised learning scheme to augment the online stage. Different from the offline stage, the substitute model in the online stage is trained with the victim output, and has similar outputs to the victim model. Therefore, the online semantic alignment scheme is aimed at further improving the performance of the substitute. Online semantic alignment uses weak augmentations for the substitute to produce pseudo labels, and aligns the substitute output of strongly augmented data with the pseudo labels. This approach gains extra accuracy increase for the substitute model over the original supervised training scheme.

Informative query generation. We also make an important contribution to the conventional model extraction framework by designing a query generation procedure with richer information. Our proposed query generation process achieves three main goals to increase the information density of synthesized queries:

1. Aggregating: the generator can effectively merge information from multiple public data,
2. Informative: the generator can produce information-extracting queries that minimize the gap between the substitute and the victim,
3. Stealthy: the synthesized queries maintain the structure of a natural image instead of collapsing into indistinguishable patterns. We propose an aggregation architecture and three loss functions for the query generator to achieve all three goals at the same time.

We have also modified our manuscript to provide an overall discussion on the design of the framework (Section 3.2), and provide more motivation for each main component of our framework (Section 3.3, Section 3.4).

3. Is it possible to provide a simpler graph than Figure 1 for illustrating the main idea?

We thank the reviewer for the suggestion. We have modified Figure 1 to provide a clearer illustration of our two main contributions. Specifically, we have simplified the illustration of our proposed aggregated query generation to highlight the novel aggregation design.

We thank the reviewer for acknowledging the technical merits of our manuscript and pointing out the problems in the presentation. We would like to make the following clarification on our contribution and make corresponding modifications to the manuscript.

1. Give explicit discussion or intuition why the proposed method can improve the query efficiency.

We appreciate the suggestion of the reviewer. Our work improves the query-efficiency of model extraction attacks by proposing a query-free self-supervised scheme and a query-efficient supervised approach. Both methods are aimed at extracting the knowledge from the public dataset effectively to reduce the query cost. We would like to discuss the motivation of each method as follows.

Query-free Self-supervised Scheme. We note that the conventional public dataset based model extraction attacks only use public data for crafting queries to the victim model. In contrast, we develop a self-supervised training scheme to optimize the substitute model exclusively based on the public dataset, which does not require any queries to the victim model. Our approach builds on the assumption that a well-trained victim model maintains semantic consistency, i.e., outputs similar representations for different images featuring the same object. Using this semantic consistency as an additional prior, our method optimizes the substitute model to learn similar encoding patterns to the victim model. We also design different variants of the scheme according to the characteristics of offline and online stages that reduce the query cost of both stages.

Query-efficient Supervised Scheme. To further improve the query-efficiency of the supervised learning stage of conventional attacks, we focus on synthesizing queries with richer information. We point out that most existing model extraction methods based on public data feature a one-to-one correspondence between the public data and the generated query, which limits the information density in the query. To increase the information diversity in the query generation process, we propose to parallelly process different public data when synthesizing a single query. We design a multi-encoder aggregation design and reconstruction loss to merge the information from different public data, and develop inconsistency and diversity loss to ensure such information is fully leveraged for minimizing the gap between the substitute and the victim models.

In our revised manuscript, we also made major revisions to the overview of our method (Section 3.2), the semantic alignment scheme (Section 3.3), and the aggregated generation method (Section 3.4), including a more detailed discussion of how our attack reduces the query cost of the model extraction process.

W2 Concerns regarding the designs of aggregated query generator

a) Is there a way to select the best base image out of $m$ images instead of using $x_{pub, 1}?$

We thank the reviewer for the valuable suggestion. We would like to point out that selecting the most suitable public data as the base image requires comparing the $m$ images based on the return of $V$, which causes extra queries and does not align with our goal of query efficiency. In such a case, there is an exploration-utilization tradeoff between high query-efficiency and finding the theoretically optimal base image. Therefore, we choose to achieve higher query-efficiency by randomly selecting a base image.

b) Why not use encoders with shared weights in the aggregated design?

This is a very insightful comment, and we have also considered the shared encoder design when designing the aggregated query generator. As shown in Tab. R2, we have found that using a shared encoder performs slightly worse than using different encoders for different indexes, and finally, we chose the latter as the current design.

Table R2. Comparisons between Encoder Architecture

A potential explanation for this observation is that the encoders for the base image (i.e., $x_{pub,1}$) and the merged images (i.e., $x_{pub,2}, ..., x_{pub,m}$) learn different parameters for encoding different inputs.

c) Explain the reconstruction loss: why use it, how to tune $\alpha_j$, and if simple aggregation operations work.

We design the reconstruction loss that optimizes the query generator to effectively merge the information from different images, and we introduce $\alpha_j$ to ensure the generated image largely resembles the base image instead of collapsing into unnatural patterns. Therefore, we set the $\alpha_1$ to be larger for the base image and $\alpha_2,...,\alpha_m$ to be smaller for the other images. Empirically, we find setting $\alpha_2=...=\alpha_m$ achieves a good performance, and did not carefully tune $\alpha$ for each image.

As for simple aggregations,
we have considered a few basic aggregation operations when choosing the aggregation design. We found that such aggregations have two shortcomings: 1) such methods produce unnatural query images that can be easily detected by the victim, and 2) such methods provide no significant improvements or even worse results when compared to the baseline. As shown in Tab. R3, we have found our proposed aggregated generator performs significantly better than simple linear combinations, such as average, MixUp (weighted average with a random probability), or CutMix.

Table R3. Comparisons among different aggregation strategies

W3. Source of empirical improvements

We demonstrate that simply using supervised pre-training is much less effective than our offline self-supervised pre-training. More importantly, we develop offline and online self-supervised training schemes according to the characteristics of the two stages, and the combination of both methods results in a strong overall performance. Finally, we note that our proposed aggregated generator not only contributes to the overall accuracy at a comparable level when compared to offline semantic alignment, but also makes a more significant boost in ASR (11.69%+) than offline semantic alignment.

W4 Section 4.5

We thank the reviewer for the suggestion, and we move more detailed quantitative results of the appendix to the main manuscript. The modified content is marked as blue in paragraph 3, Section 4.5 in the revised paper.

Miscellaneous

We thank the reviewer for checking our manuscript in detail and proposing issues for us to further improve the quality of the manuscript. We have made the following modifications:

• Section 2.1, used in-time citation: "Mosafi et al. (2019)"
• Fixed the typo in Section 2.2: "with [a] generative adversarial network"
• We have added the following description regarding transfer-based attacks in Section 2.2: "Generally speaking, we can classify black-box adversarial attacks into two categories: substitute-based, transfer-based, and query-based. First, a number of substitute-based attacks have already demonstrated the effectiveness of using the extracted substitute model as a base for launching black-box adversarial attacks. Additionally, transfer-based attacks assume the adversary can obtain a substitute model trained on the same dataset as the victim model, and focus on improving the transferability of the adversarial samples synthesized based on the substitute model. Therefore, substitute-based and transfer-based attacks are generally complementary to each other."
• Fixed the typo in Section 3.2: "a[n] aggregated"
• We denote the public dataset as $\mathcal{D}\_{pub}$ to match $x_{pub}$, which is clearer to read. We also denote the secret dataset as $\mathcal{D}\_{secret}$ in our revised manuscript.
• We use a new index $j$ for a more clear description in Section 3.2: "Here, we take the $i$-th iteration with the query number of $n_i$ as an example...the adversary obtains the $i$-th query dataset $\mathcal{Q}\_i=\{(x\_{query}^{i,j}, y\_{query}^{i,j} = V(x\_{query}^{i,j}) | j = 1, ..., n_i\}$ by querying the victim.""
• Fixed the typo in Section 3.3.1: "that share the same weight[s]"
• We have cited more papers for Siamese network architecture in Section 3.3.1.
• Fixed the typo in Section 3.4: "we propose [an] aggregated query generator"
• We have changed the notation $x_{query}^i$ to $x_{query}$. In the diversity loss, we are aimed at reducing the distance between the queries in the $i$-th iteration and all past queries. We thank the reviewer for pointing out the typo in the index.
• Fixed the typo in Section 3.5: "Second, [w]e design"
• Denoted the experimental setting for Fig.5, Tab.2, Tab.3, and Tab.4 in Section 4.1: "In Figure 5, Table 2, Table 3 and Table 4, we use CIFAR-10 as $\mathcal{D}\_{secret}$ and CIFAR-100 as $\mathcal{D}\_{pub}$ "

Q1. How does the similarity of the public dataset to the victim model’s dataset affect results?

We thank the reviewer for the question. We would like to point out that our attacks can still achieve state-of-the-art performance when the two datasets are very dissimilar in distribution. As shown in Tab. 1 of the main manuscript, we have tested the performance of different attacks when the secret dataset is CIFAR-100, and the public dataset is ImageNet. The CIFAR-100 and ImageNet datasets are collected independently, have distinctive classes without overlap, and also have different resolutions. With such different distributions between the two datasets, our attack is able to demonstrate significantly stronger performance than the existing attacks.

Q2. What if the substitute model has a different architecture from the victim model?

We kindly point out that our attack obtains better accuracy and ASR than the best-performing attack across different substitute model architectures, as shown in Tab. 3 of the main manuscript. Moreover, we have compared the performance of our attack across more substitute model architectures in Tab. A2 of our appendix.

We appreciate the reviewer for making detailed reviews and insightful comments on our work. In this response, we have made several clarifications for our work and made some modifications to our manuscript according to the valuable suggestions of the reviewer. The revised parts are marked as blue.

W1. The novelty of semantic alignment.

We thank the reviewers for the questions. We would like to make the following clarifications.

First, we point out that no existing work in the field of model extraction has explored self-supervised learning. Although the Siamese design is common in self-supervised learning, we are the first to find that the design can be effective in model extraction, and adapt it with specific modules that are suitable for both the offline and online stages of the model extraction process. Moreover, we are the first to design an offline pre-training stage for model extraction. When designing the offline stage, we note that simply using supervised pre-training method for extracting supervised models has two major disadvantages: 1) supervised pre-training makes a strong and impractical assumption that the public dataset is annotated, and 2) such scheme performs worse that the proposed self-supervised scheme. In contrast, we demonstrate the self-supervised learning approach not only eliminates the need for public annotation, but also demonstrates superior performance in extracting supervised models, as shown in Tab. R1. Additionally, we would like to clarify that knowledge distillation is not applicable for online self-supervised training. Knowledge distillation minimizes the distance between the outputs of the teacher and student model, but the two models are the same in a self-supervised setting. Therefore, we design weak and strong augmentations to introduce semantic consistency as an extra prior that optimizes the substitute model in a self-supervised manner.

Table R1. Comparisons between supervised pre-training and our proposed self-supervised pre-training

Finally, we kindly point out that we carefully design the self-supervised approach according to the characteristics of the offline and online extraction stages. Specifically, we would like to highlight the following contributions in designing self-supervised methods for two stages.

• Network architecture: In the offline stage, the adversary does not have any knowledge of the victim model, so we design a symmetric architecture for this stage. In the online stage, as the substitute model already has similar prediction scores to the victim, we design a weakly augmented branch without backpropagation to produce pseudo labels and a strongly augmented branch to learn richer semantic information. We also design different levels of augmentations accordingly for each stage.

• Feature space for semantic alignment: In the offline stage, we perform semantic alignment in a high-dimensional feature space to transfer representational information from the public dataset. In the online stage, we directly optimize semantic consistency loss on the classification scores, as the substitute predictions are relatively accurate in this stage.

I thank the authors for their responses, as well as the updates they've made to the paper. I've also read the other reviewers comments.

Despite the authors' arguments, I'm still not entirely convinced by the novelty here. The field as a whole is well-aware at this point that self-supervised learning is effective, and I don't believe that "Self-supervised Learning Applied to X" for every problem setting is necessary or novel. I understand that there are some nuances to the method beyond that, but I find these modifications relatively small. While novelty is not the sole basis of my score, it's something that I consider as part of my holistic evaluation.

I also remain concerned about the methodology. I strongly believe we should not add ordering dependencies into a method where there are none. I accept that choosing a base image without increasing query count is nontrivial (although perhaps some comparison between images offline without querying the victim may be possible). However, the use of separate encoders is not something that makes sense to me, as again, the order of the images here is completely random and does not contain any learnable patterns. Perhaps there's some tangible benefit from having a different encoder for the base image given its distinct role compared to the others, but the method could still benefit from some re-thinking to make it more permutationally invariant.

Q1: As I stated in my original question, while yes, there are some differences, CIFAR and TinyImageNet are both object-centric RGB datasets, and features will tend to transfer relatively well between them. There are much more different image datasets out there.

Q2: For the different architectures in Table 3 and A2, what is the victim model architecture?

I don't believe that applying "Self-supervised Learning Applied to X" for every problem setting is necessary or novel. I understand that there are some nuances to the method beyond that, but I find these modifications relatively small.

We thank the question of the reviewer regarding the novelty of self-supervised model extraction. We would like to clarify that it is necessary and novel to design self-supervised learning approach for model extraction. As most existing model extraction attacks only focus on information-extracting data synthesis, we point out a different approach regarding the training procedure of the substitute model. Our self-supervised approach is orthogonal to existing studies on supervised model extraction, and our proposed offline and online self-supervised methods can be combined with existing supervised learning attacks for significantly improved accuracy, ASR, and query-efficiency without extra query cost. We believe this finding is impactful on the topic of model extraction, uncovering a different research direction for this field. Furthermore, by proposing the self-supervised model extraction, we reveal an important finding that the utilization of public data can pose serious security threats to private training data and confidential models, raising concerns about the protection of AI privacy and robustness.

Additionally, we kindly point out that the modifications we made for the original self-supervised framework are novel and effective for model extraction. Particularly, we have designed the stage-specific architecture, the feature space for alignment, and different semantically equivalent branches, all of which contribute to the strong overall performance.

Although there's some tangible benefit from having a different encoder for the base image given its distinct role compared to the others, the method could still benefit from some re-thinking to make it more permutationally invariant.

We thank the reviewer for the suggestion. We have designed another architecture for the aggregated generator that both identifies the difference in the encoding of the base image and ensures the permutational invariance of the other images. Specifically, we design a base encoder for encoding the base image and a shared merge encoder to encode the other images. Also, we adopt a new formulation of the reconstruction loss with only two weight hyperparameters, one for the base image and one for the other merged images: $\mathcal{L}\_R = \frac{1}{m}(\alpha_1||G(x\_{pub,1}, x\_{pub,2}, ..., x\_{pub,m}) - x\_{pub,1}||\_2+\sum_{j=2}^m\alpha_2||G(x\_{pub,1}, x\_{pub,2}, ..., x\_{pub,m}) - x\_{pub,j}||\_2),$

Here, we assign $\alpha_1>\alpha_2$ to ensure that the generated query preserves the structure of the base image. This new formulation explicitly constrains the permutational invariance of the merged images and also gives a higher weight for the base image. The above design ensures that the structure of the base image is better preserved while encoding the other images in a permutationally invariant manner.

Table R1 shows this design obtains slightly better performance than the original design with different encoders. We have also included this design and results in Section D of our revised appendix and the new formulation of reconstruction loss in Section 3.4.2 of our revised manuscript.

Table R1. Accuracy, fidelity, and ASR of two aggregation designs

Q1: Provide the results for more different public datasets.

We thank the suggestion of the reviewer. We have compared our method with the best-performing attack, Knockoff Nets, on different public datasets. As shown in Table R2, we have found that our method generalizes significantly better than the state-of-the-art attack on different datasets. Here, the secret dataset is CIFAR-10. We have also included the results in Section J of our revised manuscript. Additionally, we are running experiments with more public datasets, which will be provided in the final version of the paper.

Table R2. Accuracy, fidelity, and ASR of different attacks across diverse public datasets.

Q2: For the different architectures in Table 3 and A2, what is the victim model architecture?

We thank the reviewer for the question. The victim model architecture is ResNet-34 in Table 3 and A2 (changed to A3 for the revised version). We have also provided this information in Section 4.1 of our revised manuscript and Section I of our revised appendix.

Although the method works great, it will be great to see some theoretical justifications behind the framework.

We really appreciate the positive comments and valuable suggestions of the reviewer. We would like to provide the following theoretical justifications for our proposed semantic alignment in this response. We have also included this content in Section C of our revised Appendix.

The key insight of our semantic alignment approach is that the adversary can optimize a substitute model $S$ on a public dataset $\mathcal{D}\_{pub}$ to learn features that are similar to the victim model $V$, which is trained on a secret dataset $\mathcal{D}\_{secret}$. Without loss of generality, we assume the substitute model $S$ is composed of an encoding function $f$ and a fully connected layer $W$. In such a case, our observation can be reformulated as follows: if $f$ has a low semantic consistency loss and $W$ is trained on $\mathcal{D}\_{secret}$ to evaluate the classification performance of $f$, $S$ has a low average classification loss on $\mathcal{D}\_{secret}$. To prove this observation, we first formally define the notations.

The encoding function of $S$ belongs to $\mathcal{F}$, a class of representation functions $f: \mathcal{X} \rightarrow \mathbb{R}^d$, such that $||f(\cdot)||\leq R$ for some $R > 0$. We denote the set of all classes in $\mathcal{D}\_{pub}$ as $\mathcal{C}$, and each $c\in\mathcal{C}$ follows a probability distribution $\mathcal{D}\_c$. The supervised classification loss of $S$ on $\mathcal{D}\_{secret}$ can be defined as $\mathcal{L}\_{\text{sup}}(S):= \mathbb{E}\_{(x,c)\in\mathcal{D}\_{secret}} \left[ l\left(S(x)\_c - S(x)\_{c'\neq c} \right) \right],$ where $l$ is a standard hinge loss or logistic loss, and $S=Wf$. When evaluating $S$, the best $W$ can be found by fixing $f$ and finetuning $W$. Therefore, we only denote the supervised loss of $f$ on $\mathcal{D}\_{secret}$ as: $\mathcal{L}\_{\text{sup}}(f) =\inf_W \mathcal{L}\_{\text{sup}}(Wf).$ Also, our offline semantic consistency loss can be formalized as

$$\mathcal{L}\_C = -\mathbb{E}\_{x\in\mathcal{D}\_{pub}}\left[{\rm log} \frac{{\rm sim}(S({\rm aug_{M}^1}(x)),S({\rm aug_{M}^2}(x)))} {\sum_{x'\in\mathcal{D}\_{pub}, x' \neq x} {\rm sim}(S(x),S(x'))} \right],$$

The loss term can be simplified as

$$\mathcal{L}\_C = \frac{1}{M} \sum_{i=1}^{M} l\left(f(x_j)^{T}(f(x_j^+) - f(x_j'))\right),$$

Here, $x_j$ and $x_j^+$ are semantically equivalent data constructed by augmentations. With the notations above, we formalize a proposition as follows.

Proposition 1. For a substitute model $S$ composed of an encoding function $f$ and a fully connected classification layer $W$, $S$ has a low average linear classification loss on $\mathcal{D}\_{secret}$ if $f$ has a low offline semantic consistency loss on $\mathcal{D}\_{pub}$.

We use a theorem proposed in [1] (Theorem 4.1 of the original paper) to prove this proposition. Let $\mathcal{S}=\lbrace x_j, x_j^+, x_j' \rbrace_{j=1}^M$ be the triplets sampled from $\mathcal{D}\_{pub}$ to optimize semantic consistency loss, $f\_{|\mathcal{S}} = \left(f_t(x_j), f_t(x_j^+), f_t(x_j')\right)\_{j\in[M],t\in[d]}\in \mathbb{R}^{3dM}$ be the restriction for $\mathcal{S}$ for any $f\in \mathcal{F}$, and we have a complexity measure with the following Rademacher average

$$\mathcal{R}(\mathcal{F}) = \mathbb{E}\_{\sigma\in\lbrace\pm 1\rbrace^{3dM}}\left[\sup_{f\in \mathcal{F}}<\sigma, f\_{|\mathcal{S}}>\right].$$

Let $\tau=\mathbb{E}\_{c,c'\sim\rho^2}\lbrace c=c'\rbrace$, and we have the following theorem[1]:

Theorem 1. With probability at least $1-\tau$, for all $f\in F$ $\mathcal{L}\_{\text{sup}} (\widehat{f}) \leqslant \frac{1}{(1-\tau)} \mathcal{L}\_C(f) - \frac{\tau}{(1-\tau)} + \frac{1}{(1-\tau)} Gen_{M}$ where $\ Gen_{M} = O\left(R \frac{R_{s}(F)}{M} + R^{2} \sqrt{\frac{\log \frac{1}{d}}{M}}\right)$ Here, we have $Gen_{M} \rightarrow 0$ as $M\rightarrow \infty$, and when $\rho$ is uniform and the number of classes $|C|\rightarrow \infty$, we have that $\frac{1}{(1-\tau)}\rightarrow 0, -\frac{\tau}{(1-\tau)}\rightarrow 0$. Therefore, when the number of sampled training triplets is large and $f$ has a low offline semantic consistency loss on $\mathcal{D}\_{pub}$, then $S$ has a low average linear classification loss on $\mathcal{D}\_{secret}$.

[1] Saunshi, Nikunj, et al. "A theoretical analysis of contrastive unsupervised representation learning." International Conference on Machine Learning. PMLR, 2019.