Stealix: Model Stealing via Prompt Evolution

We thank the reviewer for recognizing the realism of our threat model, the scalibility of our approach, and the effectiveness of our proposed proxy metric. We aim to address their concerns below.

W1. While Stealix is tested on various victim model architectures (e.g., ResNet, VGG, MobileNet), the paper does not extensively explore more complex or state-of-the-art architectures (e.g., Transformers).

We do address this in Section 5.3: Stealing Model Based on Proprietary Data, where we apply Stealix to a real-world Vision Transformer (ViT) model trained on proprietary data and demonstrate better performance than other methods.

W2. This paper assumes that the victim model only provides hard-label outputs as a defense mechanism. However, the authors does not explore how Stealix would perform against more sophisticated defenses.

Thanks for the suggestion. Most defenses such as [1, 2] perturb the posterior prediction to reduce the utility of stolen models, while keeping the predicted class (argmax) unchanged to preserve original performance for benign users. This pushes attackers to rely on hard labels, which are less informative but immune to such perturbations. Our work directly targets this setting, where attackers proactively use hard labels to circumvent the defenses. We view exploring additional defenses as complementary and will add this discussion in the revision. We are open to evaluating Stealix against defenses that the reviewer has in mind.

[1] Taesung Lee, Benjamin Edwards, Ian Molloy, and Dong Su. "Defending against machine learning model stealing attacks using deceptive perturbations." IEEE Security and Privacy Workshops (SPW) 2019.

[2] Mantas Mazeika, Bo Li, and David Forsyth. "How to steer your adversary: Targeted and efficient model stealing defenses with gradient redirection." ICML 2022.

W3. The paper would benefit from a more detailed computational cost analysis.

We appreciate the reviewer's suggestion. We have reported the runtime comparison across methods in Appendix C: Comparison of Computation Time. The results show that Stealix maintains competitive computational efficiency while outperforming other baselines.

We appreciate the reviewer's recognition of this direction of soliciting queries as promising, and we're glad that the completeness and thoroughness of our experiments came through clearly. We aim to answer the questions below.

Q1. Why the comparison with PEZ method is deferred to appendix and is conducted under only one setting? It will be great if PEZ is also included in the comparison.

We placed the PEZ comparison in the appendix due to page limits. Since PEZ is not originally a model stealing method, but a prompt tuning technique, we include it as part of an ablation study—not a baseline comparison—to isolate the impact of our proposed components: prompt refinement with victim feedback, prompt consistency, and prompt reproduction. Please see our answers to Q2 for a detailed comparison.

Q2. Could you please clearly clarify the contribution and difference of PEZ and this work?

The key difference is that PEZ optimizes prompts using only the seed image $x_c^s$, while our prompt refinement reformulates prompt optimization as a contrastive loss over a triplet $(x_c^s, x_c^+, x_c^-)$, guided by the victim model’s predictions. This enables Stealix to capture class-relevant features more effectively. Stealix further introduce prompt consistency as a proxy for evaluation, and prompt reproduction using genetic algorithms, forming a complete and victim-aware model stealing framework.

W1. While the attacker aims to steal the entire model, Alg 1 requires a specific target class. Can you clarify which setting is used and correct any inconsistency?

We apologize for the inconsistency and we will correct it in the paper. The target class is not required as we steal the entire model. Stealix iterates over all classes to collect synthetic images. We will revise Algorithm 1 to wrap Lines 3–25 in a for each class loop, and move Line 26 outside the loop with an updated description: "Train model $A$ using all class image sets $\lbrace\mathcal{X}_c^s, \mathcal{X}_c^+, \mathcal{X}_c^-\rbrace _{c=1}^K$", where $K$ denotes the total number of classes. We will also update the algorithm input accordingly to reflect that it processes all classes, rather than requiring a specified target class $c$.

W2. How large is the seed image set needed, and used in the experiments?

We use only a single seed image per class in all our experiments, as noted in Line 205 (left column).

W3. Line 129-131, it is somewhat incorrect to claim that other methods do not utlize victim model's outputs. My understanding is that PEZ also optimizes the prompt using victim's responses, isn't it?

We are sorry for the confusion: we do not claim that PEZ and other methods do not utlize victim model's outputs. More precisely, they use the victim's predictions only during attacker model training; in contrast, Stealix additionally uses them for optimizing prompts (refinement, consistency check and reproduction). PEZ, as detailed in our response to Q2, do not use victim responses to optimize the prompt, as the class of the seed image is known. We will update the paper to clarify this point.

W4. Does the proposed method apply to stealing regression models?

Stealix can potentially be extended to regression tasks. For example, during prompt refinement, low regression error, such as a low mean square error (MSE), could be interpreted as “positive” feedback and high error as “negative,” similar to our classification setting. This would allow the triplet-based optimization and prompt consistency metric to operate analogously.

We thank the reviewer for appreciating our strict black-box threat model and recognizing the novelty of our automatic prompt construction approach for model stealing. We answer their questions below.

Q1. In Algorithm 1, are you performing Stealix with only a single class? Is that really effective? I would be interested in seeing the performance when stealing the model with samples from multiple classes.

We apologize for the mistake in Algorithm 1 and we will fix it in the revised paper. To clarify, Stealix considers all classes simultaneously. Algorithm 1 illustrates the process (Line 3-25) for a single class, but in practice, it is applied to all classes in parallel. After processing all classes, the generated images are collected and used to train the attacker model A, as outlined in the method overview (Lines 201–203, left column)

We will revise Algorithm 1 to wrap Lines 3–25 in a for each class c loop, and move Line 26 outside the loop with an updated description: "Train model $A$ using all class image sets $\lbrace\mathcal{X}_c^s, \mathcal{X}_c^+, \mathcal{X}_c^-\rbrace _{c=1}^K$", where $K$ denotes the total number of classes. We will also update the algorithm input accordingly to reflect that it processes all classes, rather than requiring a specified target class $c$.

Q2. Wouldn't initial empty $\mathcal{X}_c^+$ and $\mathcal{X}_c^-$ result in empty initial $\mathcal{S}^0$?

We will revise the algorithm to fix the notation. While $\mathcal{X}_c^+$ and $\mathcal{X}_c^-$ are initially empty, the seed set $\mathcal{X}_c^s$ is not, so initial $\mathcal{S}^0$ is populated using seed images: $\mathcal{S}^0 = \lbrace(x_c^s)_i^0\rbrace _{i=1}^N$. For generality, we denote $\mathcal{S}^t = \lbrace(x_c^s, x_c^+, x_c^-)_i^t\rbrace _{i=1}^N$ and allow any of the elements in the triplet to be null. Our prompt optimization supports learning from as little as a single image (Equation 3), ensuring the process works in the early stages where positive and/or negative samples are unavailable.

Q3. The query budget comparison in Table 1 may not be fair. Unlike other methods listed, the proposed Stealix method requires further querying the victim model during the prompt-constructing stage, as it needs to repeatedly query the victim model with newly synthesized images (see Lines#19-20 in Algorithm 1). I suggest the authors provide the exact equation for calculating the overall query budgets and list all related hyperparameters in a single table for clarity.

Thanks for the suggestion. We clarify that Table 1 presents a fair comparison, because all synthesized images in Lines#19-20 of Algorithm 1 are included in training the attacker model, with each prompt synthesizing $M$ images (Line 15 in Algorithm 1). The full query budget corresponds exactly to the queries made during prompt construction, as noted in Line 200 (left column).

Taking CIFAR-10 in Table 1 as an example, with a total budget per class of $B=500$, Stealix uses $M=10$ queries per prompt (Line 15 of Algorithm 1), resulting in 50 prompts per class. Across 10 classes, this leads to a total query budget of $10\times 500=5000$ that are used to train the attacker model, which is the same budget used for other methods.

Q4. The experiments only consider a single victim model backbone (i.e., ResNet-34), which I think is insufficient to demonstrate the effectiveness of Stealix. I suggest including additional experiments on ResNet-like/CLIP-like victim backbones.

We clarify that our paper does include comparisons across multiple victim model architectures. Specifically, Appendix H provides results with different victim backbones, and Appendix G covers variations in attacker model architectures. For both setups, we study two ResNet variations, one VGG and one MobileNet. Additionally, in Section 5.3, we demonstrate Stealix's effectiveness against a Vision Transformer (ViT)-based victim model trained on proprietary data.

Q5. In Algorithm 1, Lines#9-11 are redundant and can be removed.

Removing Lines 9–11 would allow the consumed budget $b$ to exceed the total allowed budget $B$, since $b$ is updated within the inner loop (Line 17). Nevertheless, we appreciate the reviewer's suggestion and will revise the algorithm to test the budget constraint only once.

We thank the reviewer for recognizing our contribution in addressing a key limitation of prior work, and for their appreciation of our method's clarity, effectiveness, and thorough evaluation. We answer the questions below. The reviewer can try the provided prompts at Stable Diffusion 2.1 demo. Note that the demo may use a different generation setup from ours in the experiments.

Q1. Can authors provide some examples of optimized prompts?

We provide examples of optimized prompts in the following table and will include them in the revised paper. The table showcases prompts corresponding to the high Prompt Consistency (PC) values as described in Appendix E. One immediate observation is that the optimized prompts are not always interpretable to humans, echoing our motivation that human-crafted prompts may be suboptimal for model performance. Moreover, our approach often supplements class-specific details that may be overlooked by humans. For example, gps crop emphasizes geospatial context for AnnualCrop, jungle suggests dense vegetation for Forest, and floodsaved, port, and bahamas convey water-related cues for River and SeaLake. These examples illustrate how Stealix uncovers latent features that the victim learns.

Q2. Additionally, can you give some examples how the prompts change during the evoluation. This would shed more light into why stealix works so well compared to human-crafted prompts

We provide an example to illustrate prompt evolution. In Figure 5, the seed image for the "Person" class includes a prominent dog, leading to the first-generation prompt — "chilean vaw breton cecilia hands console redux woodpecker northwestern beagle sytracker collie relaxing celticsped" — which generates dog images and results in prompt consistency (PC) of 0. Stealix then uses the misclassified image as a negative example and refines the prompt to — "syrian helene pasquspock hands thumbcuddling sheffield stuck smritihouseholds vulnerable kerswednesday humormindy intestin" — removing dog-related features and achieving PC = 1. This example shows how Stealix evolves prompts by filtering out misleading features using victim feedback. We will revise the paper to include this example for better clarity.

Q3. What is the author's intuition why the attack works so much better than the baselines for a simple dataset like Cifar-10? What do the prompts look like in this case?

Thank you for the question. As discussed in the Diversity comparison (Line 377, right column) and shown in Table 3, the better performance stems from the greater diversity in our synthetic data, enabled by prompt evolution. E.g., one optimized prompt for the cat class — "punisher desktop kittens siamese beef twins personality bosnicorgi schnautuxedo 일tuxedo satellite consecutive desktop" — includes fine-grained categories such as "siamese" and relational cues like "twins" to encourage multiple distinct instances. Additionally, terms like "desktop" provide varied contextual environments. In contrast, simple prompts like "a photo of a cat" tend to produce less diverse images. These diverse prompts are generated automatically by Stealix, without requiring a human in the loop.

Q4. Do you think the same attack method/philosophy would work for other kinds of models? (e.g. image segmentation, text classification).

It might be possible to generalize to other tasks. For image segmentation, recent work [1] shows that prompts can guide image generation toward specific segmentation layouts, suggesting that prompt refinement based on mask consistency could be a feasible direction. For text classification, the image generator could be replaced with a language model, with prompts optimized to generate inputs aligned with what the classifier has learned.

[1] Yumeng Li, Margret Keuper, Dan Zhang, and Anna Khoreva. "Adversarial supervision makes layout-to-image diffusion models thrive." ICLR 2024.

Q5. How would stealix perform on datasets with even more classes?

Stealix is expected to scale to more classes. We optimize prompts per class and treat all non-target classes as negatives, ensuring focus on class-specific features. Unlike baselines that generate images without considering predicted classes, Stealix actively steers synthesis toward the intended class.