We thank the reviewer for their valuable suggestions. We are glad they enjoy reading our paper, find the method novel, and think the insights in our analysis interesting. We would like to address the following questions.

W1. The method primarily relies on existing work on genetic optimization and hard prompt optimization. While combining these methods is novel in the domain of model stealing, the individual parts already exist in literature. I do not find this compellingly flawed but I want to mention that the paper proposes no novel, technical contributions besides combining existing methods.

We appreciate the acknowledgment of our method as a novel contribution to model stealing. While we use genetic and hard prompt optimization, each component of our method addresses a unique challenge unexplored in previous works, as discussed in the general response G1.

W2. Why should the attack be interested in minimizing the cross-entropy loss between the victim model and the surrogate model under label-only access?

We use the cross-entropy loss as a proxy for the predicted label (0-1 loss), in the same way that this former is traditionally used for training classification models as a differentiable approximation to counting model errors (0-1 loss here as well). We do not find this practice misaligned with using accuracy as a performance metric (Tab. 1). It is also a standard practice, including in the model theft community [1, 2]. Although alternative criteria may be worth exploring, such investigations are beyond the scope of this paper.

W3. The experiments seem to be conducted only once per setup.

We did conduct the experiments with three different random seeds (see confidence intervals in Fig. 3 and 6). All accuracy and corresponding PC values in the tables represent the mean across the three repetitions. We will update the paper to clearly state the number of experiment repeatitions.

W4. L144: Using the term "label-only access" might be a more explicit description of the setting compared to "black box."

Thanks for the suggestion. We will update the terminology in the paper for clarity.

W5. ... (The attack goal) becomes clearer later in the paper, but a more precise definition early in the paper would be helpful for understanding.

The attack goal is to "maximize its accuracy on a test set" as the reviewer mentioned. We will give this definition earlier.

Q1. Is there an explanation why the knowledge distillation accuracy in Table 1 is, in some cases, substantially lower than the victim model's test accuracy?

For the knowledge distillation (KD) setting in Tab. 1, the attacker has only 500 queries per class and hard label access, rather than the entire victim dataset. As shown in Fig. 3, performance improves with a larger budget. For PASCAL, the gap arises because the victim model uses ImageNet-pretrained weights for ResNet34, while the attacker does not.

Q2. How are the qualitative samples from Fig. 5 picked? Are they cherry-picked or randomly selected?

For Stealix, samples are randomly picked from images generated by prompts with higher prompt consistency that we optimize. For others, they are randomly picked.

[1] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019

[2] Vlad Hondru and Radu Tudor Ionescu. Towards few-call model stealing via active self-paced knowledge distillation and diffusion-based image generation. arXiv preprint arXiv:2310.00096, 2023.

We thank the reviewer for their insightful feedbacks. It is encouraging that the reviewer finds our method novel, particularly in enhancing image generation effectiveness and diversity, as well as in removing the need for human-crafted prompts. We aim to answer the concerns below.

W1.1. If the attacker’s objective is to steal general knowledge from a smaller model, the large foundation models already possess advanced capabilities that might surpass what a smaller model like ResNet-34 could offer.

While we agree that foundation models have zero-shot classification capabilities, they cannot surpass task-specific models in many general applications. Specifically, EuroSAT dataset in our evaluation is an example where zero-shot CLIP underperforms compared to a task-specific model, as noted in the CLIP paper [1]. Other limitations are also well-documented, e.g., in OpenAI’s blog:

1. Sensitivity to wording: Achieving accurate performance with zero-shot CLIP requires careful prompt engineering.
2. Limited generalization: CLIP struggles with abstract tasks and images not seen during training, such as counting objects or estimating distances, like how close a car is in a photo.
3. Challenges in fine-grained classification: It faces difficulty distinguishing fine-grained differences, such as between car models, aircraft variants, or flower species.

W1.2. The lack of domain-specific knowledge in CLIP and diffusion model (DM) leads to lower performance.

While we agree that domain-specific knowledge might be lacking in pretrained CLIP and DM, their ability to generate novel concepts might be mainly constraint by the user's ability to describe the desired target through text, as stated in Textual Inversion [2]. This statement is consistent with our method and results: with an appropriate prompt optimization objective (for us, prompt consistency), DM can significantly increase the risk of model stealing compared to previous methods.

W1.3. If it’s feasible to extract knowledge from a larger model?

Yes, it is feasible. Our method does not rely on the victim model's size, as shown in Appendix D and E across various attacker and victim architectures and sizes. If the reviewer is referring to larger datasets rather than to larger model size, it is still feasible as the attacker increases the query budget and the surrogate model size if necessary. We are glad to address further concerns if the reviewer suggests something else.

W2. ...(one image per class) may actually provide more information than a class name alone ...it seems that the primary novelty over previous work lies in using CLIP to automatically generate prompts rather than manually crafting them.

While one image per class might provide more information, relying on seed images and CLIP does not guarantee effective prompts for model stealing. When prompts are optimized solely on the seed image using CLIP, accuracy drops significantly (Tab. 5, "Stealix w/o reproduction"), e.g., from 40.0% to 26.7% in PASCAL, highlighting the importance of our work. Moreover, even with the seed image and class name, prompts generated by InstructBLIP (built on CLIP) fail to synthesize accurate EuroSAT images (Fig. 4), leading to lower attacker model accuracy (Tab. 2). The clarification of the one-image-per-class setup is in the general response G1 and our novelty in G2.

[1] Radford, Alec, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry et al. "Learning transferable visual models from natural language supervision." In International Conference on Machine Learning (ICML), 2021.

[2] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In International Conference on Learning Representations (ICLR), 2023.

We thank the reviewer for the insightful and positive feedback. We are glad the reviewer found our method novel, our proposed threat model realistic, and the paper well-written. We address the questions below.

Q1. What distinguishes Stealix's Prompt Refinement from Wen et al. (2024)'s approach? The performance improvement seems to come primarily from Prompt Refinement.

The main difference lies in the use of triplet images to capture features learnt by the victim model. Specifically, Wen et al. (2024) optimize prompts with the seed image $x_c^s$, while ours integrates the victim model’s predictions with a triplet $(x_c^s, x_c^+, x_c^-)$ in the prompt refinement. This highlight positive features and discard negatives learnt by the victim model. We clarify that 'Stealix w/o reproduction' in Table 5 refers to using Wen et al.'s method with only $x_c^s$ to train the attacker model. Their method (Tab. 5, Stealix w/o reproduction) results in a low performance (e.g. 26.7% on PASCAL) compared with ours (40.0%), demonstrating the effectiveness of our prompt refinement and reproduction.

Q2. Could you explain in more detail what is meant by "one-shot setup" in the Method section, lines 214-215? Does the attacker need to know at least one class from the victim model, or can it work without any class information? Additionally, what are the implications of this one-shot setup on the method's practicality, and how does it compare to other approaches in terms of required prior knowledge?

We borrow this terminology from existing literature, such as DA-Fusion [1], which refers to a setting where there is one image per class ($|X_c^s|=1$) available. In practice, end-users, like competitors, often have limited images at hand, which justifies the setting and motivates the attack. For additional clarification on why this setting is realistic, we refer the reviewer to general response G1. In Table 1, we compare our approach closely with various baselines, including DA-Fusion and Real Guidance, both of which also rely on the one-image-per-class setting.

[1] Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov. Effective data augmentation with diffusion models. In International Conference on Learning Representations (ICLR), 2024.

Q3. Source code not available.

We are committed to making the code publicly available upon acceptance, as noted in the Reproducibility Statement.

W1. Lack of details on t in generating S in Algorithm 1. The paper could benefit from a more detailed explanation of how t is set or updated during the generation of S.

$t$ is the population index, as defined in line 3 and updated in line 23 of Algorithm 1.

W2. Missing experiments on Prompt length (L).

Thanks for the suggestion. We additionally evaluate the impact of different prompt lengths (4, 16, and 32) on EuroSAT, using a query budget of 500 per class across three random seeds. The table below presents the mean accuracy of the attacker model. Stealix consistently outperforms other methods (best baseline is 59.0% from DA-Fusion) across all prompt lengths. The observed differences in performance further justify the choice with prompt length 16, balancing efficiency with accuracy.

W3. Lack of time comparisons... In Appendix A, line 662, the authors mention "to save optimization time," but a more detailed explanation of the actual time saved or required would be beneficial.

We appreciate the suggestion and provide a time comparison with EuroSAT as an example. All experiments were run on a single machine equipped with an NVIDIA V100 32GB GPU and an AMD EPYC 7543 32-Core Processor (Line 320-321).

For the prompt refinement, each prompt requires ~18 seconds for 500 optimization steps, compared to ~3 minutes for 5000 steps.

Additionally, we present the time taken for the entire process, using a 500-query budget per class (DFME uses 2M queries per class). The table below shows that Stealix achieves state-of-the-art accuracy with a reasonable computational time.

We have updated the paper with the suggestions made by the reviewer: Impact of prompt length in Appendix A marked as blue; "Comparison of Computation Time" in Appendix I.

If the concern has been properly addressed, we would kindly request the reviewer to reconsider the score.

Q9. Are the prompts randomly initialized at every iteration t?

Yes, they are. As shown in Algorithm 1, Line 12, the PromptRefinement function does not accept previous prompts. This initialization is also stated in Line 3 of Algorithm 2.

Q10. Line 4 of the algorithm says that the last two elements of the image triplets are randomly sampled from the sets X_c^+ and X_c^-. But these two sets are initialized as empty sets in Line 3. How do you draw samples from empty sets?

Sampling from empty sets means no samples are selected, thus the population starts with only seed images.

Q11. Line 16 of Algorithm 1 claims that Line 17 and Line 18 consume the query budget. But the budget should have already been consumed in Line 15.

We clarify that Lines 17 and 18 update the sets, and Line 19 updates the variable that tracks the consumed budget, instead of consuming the query budget, as they are consumed in Line 15.

Q12. What does "CIFAR10 is a standard classification dataset, where class names can be treated as ground truth for synthesizing relevant images." mean in Section 5.1? Do you mean that you use the class names in CIFAR10 experiments? If so, it contradicts the earlier claims in the paper.

No, our method does not use CIFAR-10 class names or any other class names. We mean that class names of CIFAR10 can sufficiently act as a guide for relevant image synthesis, thus methods like Real Guidance using class names of CIFAR10 is a strong baseline.

Q13. Line 460 says "retain the positively labeled images to form the transfer dataset for training the attacker model.". Could the authors elaborate on it more? I do not understand this sentence.

We only use positive images ($X_c^+$) to train the attacker model in this analysis, differing from the main setting (Algorithm 1). This experiment examines how maximizing the PC improves attacker model performance. For instance, the victim model classifies images prompted with "black dog" and "white dog" as "dog," but "white dog" has a higher PC. We demonstrate that higher PC enhances model performance (see Fig. 6). Using only positive images ensures that 'dog' images are generated strictly from 'dog' prompts (e.g., 'black dog' or 'white dog') rather than unrelated prompts for other classes like 'white cat.'

Q14. Line 446 claims that "Since the attacker lacks access to victim data", but the paper assumes that one image per class is available to the attacker.

One image is not the same as a full dataset or distribution in terms of realism and feasibility for the attacker. To improve clarity, we will update this claim to: 'Since the attacker lacks access to the distribution of the victim data.'

Q15. The paper claims in multiple places that their generated data distribution aligns with that of the victim data ... Note that "align the distribution" has a mathematical implication and it can mislead the readers to think that you are minimizing it ... I would suggest the authors to rephrase these claims.

Thanks for the suggestion, we will replace the use of this term.

We thank the reviewer for the thoughtful feedback and considering our results promising. We now address the concerns below.

Q1. Regarding class names... Could the authors provide an example of a real-world classification service where class name information is not known to end users?

We clarify that we do not mean that class names are unavailable, but might be insufficient to generate images similar to the victim data, such as EuroSAT shown in Fig. 4 and 5. We limit class name use just because they may by chance serve as good prompts for dataset like CIFAR10, while we allow baselines like Real Guidance to use both class names and seed images, demonstrating our approach outperforms others across threat models.

Q2. Regarding one image per class ... I would regard this as a huge constraint...such as medical image diagnosis systems.

Model stealing attack could be launched by end-users like competitors within the same field, as discussed in general response G1. For instance, in the medical domain, competitors may have limited medical images, but restrictions like high costs and privacy hinder their scalability, driving them to steal models with the limited images.

Q3. ... If the attacker has access to one image per class, they can directly look at those images and might be able to have a good guess of the class name.

As mentioned in Q1, the class names are available but likely fail to describe the victim data accurately. Moreover, prompt engineering is labor-intensive and non-trivial, which often limits scalability (L15-16). In contrast, our approach provide an automatic and scalable solution. To further investigate the limitations of prompt engineering, we adopt cutting-edge vision-language model, InstructBLIP, in analysis "Limitations of human-crafted prompts" (Line 374). Given the class name and seed image, the prompt designed by InstructBLIP fails to synthesize EuroSAT images in Fig. 4. It indicates that seed images associated with their class names are not sufficient to synthesize victim data, which necessitates our approach.

Q4. The paper should list clearly which baseline methods require real images (and how many) and which do not.

Table 1 has listed the requirements of different approaches, including the number of real images and the use of class names. We will explicitly highlight this for better clarity.

Q5. ... augment the images with approaches such as DA-Fusion, and then train the surrogate model directly.

While data augmentation without querying the victim model is not considered as model stealing, we include an additional experiment comparing attacker model accuracy in model stealing and data augmentation setup suggested by the reviewer (last line) with 500 query budget per class. The table below shows that performance degrades significantly with DA-Fusion when relying solely on class labels for training, highlighting that model stealing is essential, even with one image per class.

Q6. ... How do you define the fitness value for the triplet?

The fitness value of a triplet is the value of its associated prompt, which is derived through prompt refinement with triplet images (Equation 3).

Q7. ... What do you mean by "split at a random point" and "segments" for image triplets?

It means randomly exchanging images between two triplet sets to create a new set. For example, given two triplets $(x_1, x_2, x_3)$ and $(x_4, x_5, x_6)$, a new triplet could be like $(x_1, x_5, x_6)$. We will update the phrasing for clarity.

Q8. Will the negative samples in line 25 of Algorithm 1 be used to train the surrogate model? If so, do you use their raw label from the victim model as the training label, or simply minimize their prediction prob of class c? This is not described in the paper.

Yes, they are used to train the surrogate model, as stated in Line 213-215, where $B$ contains both positive and negative images. We use the raw label from the victim model, as described in the Equation (2). We will highlight this in the paper.