Proposition 1 provides meaningless differential privacy protection ($\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$).

Thank you for your feedback. We assume there may be a misunderstanding about the role of random sampling in our method that we'd like to clarify. We will interpret the significance of Proposition 1 from the perspectives of what, how, and why. Finally, we will address the reviewer's concern about $\delta$ with targeted experiments.

1. Comprehensive privacy analysis of the distilled dataset: We would like to emphasize that our privacy analysis of the distilled dataset is a holistic process that involves two stages, as defined in Line 182 of our paper. Proposition 1 pertains solely to the privacy analysis of the initial random sampling phase. The privacy guarantees of the distilled dataset are further analyzed in Theorem 1 and Theorem 2, where we use the Kullback-Leibler divergence between neighboring datasets to measure the privacy of the distilled dataset. Our findings indicate that the distilled dataset exhibits properties akin to differential privacy.
2. Clarification on the Value of $\delta$, which motivates us to focus on the privacy leakage of initialization samples
   1. We acknowledge the concern raised regarding the value of $\delta$ obtained in Proposition 1, which is $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$. This value indeed exceeds the threshold of $\frac{1}{|\mathcal{T}|}$ as described in Dwork's privacy book. However, it is important to note that we do not directly use the randomly sampled subset of real data as the distilled dataset, as this would indeed compromise the privacy of the sampled data.
   2. In Proposition 1 and Remark 1, we explicitly state that $|\mathcal{S}|/|\mathcal{T}|$ represents the proportion of real privacy data selected for initialization. This proportion indicates that the privacy of these initial samples cannot be fully protected through the fluctuations in the matching optimization process. This realization motivated us to introduce the concept of explicit privacy and the KT plugin to safeguard the privacy of these initialization samples.
   3. Dwork's assertion that values of $\delta$ on the order of $1/\Vert x\Vert_1$ are dangerous stems from the "just a few" philosophy, refering to the last paragraph on page 9 of Dwork's privacy book [1] . The book states ”For a single data set, ‘just a few’ privacy can be achieved by randomly selecting a subset of rows and releasing them in their entirety”, where the privacy of a very small number of samples is directly compromised. In our work, the introduction of explicit privacy and the KT plugin is precisely aimed at addressing this issue, rather than being a weakness or flaw. The dataset distillation process introduces perturbations from other samples during the matching optimization phase, but when the IPC is high, these perturbations are insufficient to protect explicit privacy, as illustrated in Figure 1. The KT plugin, therefore, applies perturbations during the initialization phase to protect explicit privacy while reducing the KL divergence between neighboring datasets.
3. In addition to revealing that random sampling can lead to privacy leakage of initialization samples, Proposition 1 effectively protects non-initialization samples: Proposition 1 is merely the first stage of the dataset distillation process. We encourage the reviewer to consider the entire framework of dataset distillation. Proposition 1 effectively compresses the sample space, thereby significantly protecting the privacy of the majority of the data and effectively defending against membership inference attacks. For the privacy of the initialization samples, we provide protection through the matching optimization process and the KT plugin.

4. Experimental Validation of Initialization Sample Privacy: We have conducted experiments to address the concerns regarding the privacy of the initialization samples, as raised by both you and Reviewer rmpC. In Table 4 at Line 1053 of our paper, we specify the initialization samples for a fix target membership inference attack [2].
   • Notably, the success rate of membership inference attacks on initialization training samples decreased from 99.5% to 54.1%. The experimental results demonstrate that the introduction of the KT plugin effectively makes the privacy information of the initialization samples less susceptible to membership inference attacks, thereby protecting their explicit privacy and effectively defending against such attacks.

We hope these clarifications address your concerns and provide a more comprehensive understanding of our approach to privacy protection in dataset distillation. Thank you once again for your valuable feedback.

[1] Dwork, Cynthia, and Aaron Roth. "The algorithmic foundations of differential privacy." Foundations and Trends® in Theoretical Computer Science 9.3–4 (2014): 211-407.

[2] Ye, Jiayuan, et al. "Enhanced membership inference attacks against machine learning models." Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2022.

Dear Reviewer 4Amd,

Thank you once again for your insightful feedback on our submission. We would like to remind you that the discussion period is concluding. To facilitate your review, we have provided a concise summary below, outlining our responses to each of your concerns:

• Proposition 1 is the motivation for our proposed explicit privacy, not a flaw.
• Explicit Privacy and Differential Privacy: Differential privacy protects membership identity, whereas explicit privacy safeguards against privacy leakage in data release (as illustrated in the new PDF, Figure 1). Theoretically, we analyze the leakage of individual samples through KL divergence. In our experiments, we assess membership inference attacks and explicit privacy on distilled data, comparing the original dataset and KT plugin results. The KT plugin significantly enhances explicit privacy and defense against inference attacks while preserving original distillation effects.
• A Comprehensive Revision: We have revised the title and the main text to avoid overclaiming, added scene images and descriptions to better understand the distillation data scenario, clarified the relationship with differential privacy, and reorganized the contributions of this paper.

More detailed information can be found in the general response and the revised PDF. We are grateful for your insightful comments and are eager to confirm whether our responses have adequately addressed your concerns. We look forward to any additional input you may provide.

Warm regards,

The Authors of Submission 10266.

W2&Q2: $\delta$ would need to be less than or equal to $n^{-1}$. Any relaxations or modifications that could make it more feasible?

We would like to emphasize that we use Proposition 1 to motivate the explicit privacy leakage problem and the design of our KT plugin, and our proposal consists of two stages: (1) random training data sampling for initialization, and (2) matching optimization with randomness.

• Proposition 1 reveals training data privacy used for initialization
  • In Proposition 1, we obtain $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$. This value is greater than the differential privacy requirement of less than $\frac{1}{|\mathcal{T}|}$, indicating that the privacy of the $|\mathcal{S}|$samples used for initialization is compromised. This is precisely the motivation for our explicit privacy leakage.
  • Based on our analysis in Remark 1, we believe that even after the matching optimization process (Theorem 1), the samples used for initialization in high IPC distillation methods still face significant privacy leakage risks.
  • Therefore, we propose the KT plugin to protect the privacy of these initialized samples. This should not be seen as a weakness but rather as the privacy risk inherent in dataset distillation methods that use real data for initialization.
• Privacy analysis of dataset distillation in two stages yields privacy protection properties of the final distilled dataset
  • It is important to note that Proposition 1 pertains only to the first stage of dataset distillation, not the final optimized distilled dataset. Proposition 1 not only reveals the leakage of initialization training samples but also demonstrates the effective protection of training samples not used for initialization, thereby effectively defending against membership inference attacks.
  • The complete privacy analysis of dataset distillation requires reference to Theorem 1 and Theorem 2 after the introduction of the KT plugin. In the second matching optimization stage, we analyze the properties similar to differential privacy using the KL divergence of adjacent datasets. At the same time, the optimization brought by matching is considered as privacy protection for initialization training samples. As seen in Figure 1, the information of the initialized data is protected at low IPC. We emphasize the use of the KT plugin to address the issue of explicit privacy leakage at high IPC.
• Validate the above discussion with experiments
  • Existing dataset distillation methods do not consider the explicit privacy leakage risks associated with high IPC. The random real data sampling with $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$is particularly noteworthy for the privacy of initialization samples. Hence, we propose the KT plugin to protect the privacy of initialization samples.
  • We implemented a fix-target membership inference attack [1] on line 472 (table 3) specifically to attack the privacy of the initialized data. After using the KT plugin, the initialized sample data can be protected against MIA.

[1] Ye, Jiayuan, et al. "Enhanced membership inference attacks against machine learning models." Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2022.

W3: The comparison with differentially private generators appears to be unfair.

We appreciate the reviewer's concern about ensuring fair privacy comparisons. We will first explain why the KT plugin is compared with DP-generators, then justify the choice of $\epsilon=10$ for comparison, and finally, we will supplement experiments to create trade-off curves to maintain fairness.

• KT can achieve comparable privacy protection capabilities to DP-generators while focusing on the usability of the distilled dataset: In the previous response, we discussed the privacy protection of the entire dataset distillation process and the enhanced privacy protection provided by the KT plugin, which is based on the KL divergence analysis of adjacent datasets. This demonstrates that the distilled dataset possesses properties similar to differential privacy. Although the KT plugin cannot directly provide privacy guarantees, it can be compared to DP-generators in terms of their defense against membership inference attacks. It also reflects that KT-DATM significantly enhances the usability of data under limited IPC conditions, a capability that existing DP-generators cannot achieve.
• $\epsilon=10$ is already a relatively relaxed condition for DP-generators: We originally presented results only for $\epsilon=10$ because we emphasize that the information aggregation capability of existing DP-generators is far inferior to that of dataset distillation methods. An $\epsilon$ value of 10 already represents a relatively relaxed privacy protection.
  • If $\epsilon$ were to be set even smaller, the usability of the data would be significantly worse. Our KT method achieves the best data information aggregation effect under similar MIA defense and explicit privacy protection.
• Add trade-off curves for fair comparison: We should present more comprehensive results to readers to facilitate a better understanding of the distinctions between dataset distillation with the KT plugin and DP-generators. We have created trade-off curves in the appendix for DP-generator methods using different $\epsilon$ values in line 460 on page 9 and the Appendix G (in line 1016 on page 19). Notably, for methods like PSG, even as $\epsilon$ approaches infinity, their data utility remains inferior to KT-DATM.

We would like to thank you for your time spent reviewing our paper and for raising questions about the validity of Proposition 1 and the fairness of the comparison with the DP-generator. Below, I will address these concerns separately.

W1&Q1: Difference between random sampling for initialization and privacy amplification via sub-sampling. Why does the random sampling method in data distillation provide differential privacy guarantees without prior privatization?

To address your question comprehensively, we will first explain the mechanism of random sampling for initialization in our method, then demonstrate the correctness of its privacy guarantees through theoretical analysis, and finally elucidate its key distinctions from privacy amplification via sub-sampling.

• Random sampling for initialization

  As mentioned on line 182, we divide the data distillation process into two stages: the initialization stage and the matching optimization stage. Proposition 1 analyzes only the first stage, where samples are randomly selected to initialize the distilled dataset, and concludes that it satisfies $(\ln\frac{|\mathcal{T}|+1}{|\mathcal{T}|+1-|\mathcal{S}|},\frac{|\mathcal{S}|}{|\mathcal{T}|})$-DP. However, to analyze the privacy protection of the entire distillation process, one must refer to Theorem 1 and Theorem 2 after the introduction of KT, particularly how the KT plugin and subsequent optimization process protect the private training data corresponding to the initialized distillation samples.

• Proposition 1 is correct and it justifies the limitation that dataset distillation methods with real data initialization would leak the privacy of corresponding initialized real samples.

  • We provide complete proof of Proposition 1 in lines 792 on page 15 (Appendix B), demonstrating the differential privacy properties of random selection according to the definition of differential privacy.
  • We can also understand this from the perspectives of $\epsilon$ and $\delta$. $\epsilon$ represents the scaling value of the probability of obtaining the same subset, while $\delta$ represents the probability of failing to obtain the same subset, i.e., the probability of privacy protection failure. Our obtained $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$, which is precisely the probability of selecting the differing sample in adjacent datasets. If the differing sample is selected as a subset member, privacy protection fails.
  • However, $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$ can also be interpreted as leading to the privacy leakage of the selected samples, a point raised in your W2, so that we propose KT plugin. We will further address this in W2.

• Difference from privacy amplification via sub-sampling

  The distinction between our approach and privacy amplification via sub-sampling lies in their fundamental mechanisms and objectives:

  • Our random selection without replacement serves as an integral component of the data initialization process, where Proposition 1 establishes the privacy guarantees specifically for the unselected samples. While this mechanism may potentially expose the privacy of initialized samples (as we will address in the subsequent response), its primary purpose is to establish baseline privacy properties during the initialization phase.
  • In contrast, privacy amplification via sub-sampling operates as a post-processing technique that enhances existing privacy protection mechanisms. It achieves this by reducing the effective privacy budget through random sampling, thereby strengthening the overall privacy guarantees of the original mechanism.

Dear Reviewer rmpC,

Thank you for your thoughtful feedback and for acknowledging the clarification regarding Proposition 1. We greatly appreciate your careful review and the time you've spent understanding our work more deeply.

We have made substantial revisions to improve the clarity of our presentation, particularly focusing on (1) contributions in lines 102-112 on page 2 and (2) statement of Proposition1 in lines 237-252 on page 5.

• Our revised contributions emphasize that we are the first to propose explicit privacy (in lines 102 on page 2), as well as the theoretical analysis of the root cause of the explicit privacy leakage and the weakening defense against membership inference attacks as IPC improves, which is due to the random initialization of training sample sampling (in lines 104 on page 2).
• Claims on Proposition 1 in lines 237-252 on page 5 now clearly point out that the differential privacy achieved through initialization is unreliable, with $\delta=\frac{|\mathcal{S}|}{|\mathcal{T}|}$ serving as a clear indicator of initial data privacy leakage. This observation directly motivates our introduction of explicit privacy protection.
• Furthermore, in line 253 on page 5, we emphasize that the perturbations applied to initialized distilled samples in Phase 2 are insufficient for privacy protection. This crucial point is reinforced in Remark 1 (line 280), providing a more coherent narrative of our contributions.

These revisions (highlighted in blue in the updated PDF) aim to prevent similar misunderstandings and present our explicit privacy findings more clearly. We appreciate your increased confidence in our work and remain open to any additional suggestions for improvement.

If our revision has addressed your concern, we kindly request you to re-consider your score. We greatly value your assessment and believe our improvements align with your constructive feedback.

Dear Reviewer rmpC,

Thank you once again for your insightful feedback on our submission. We would like to remind you that the discussion period is concluding. To facilitate your review, we have provided a concise summary below, outlining our responses to each of your concerns:

• Understanding Explicit Privacy: Differential privacy can protect membership identity, but it does not guarantee the prevention of privacy leakage issues. Therefore, in the context of data release, it is necessary to protect the data itself. Currently, the release of distilled data has exposed the privacy information of initialization samples. The use of the KT plugin in distilled data can simultaneously defend against both explicit privacy leakage and membership inference attacks.
• A Comprehensive Revision: We have revised the title and the main text to avoid over-claiming, added scene images and descriptions to better understand the distillation data scenario, clarified the relationship with differential privacy, and reorganized the contributions of this paper.

More detailed information can be found in the general response and the revised PDF. We are grateful for your insightful comments and are eager to confirm whether our responses have adequately addressed your concerns. We look forward to any additional input you may provide.

Warm regards,

The Authors of Submission 10266.

Q1: What would happen if you used random initialization or added noise to the real image selection process for initialization?

We appreciate the reviewer's suggestion to use random initialization or adding noise instead of random real data sampling for initialization. Initially, dataset distillation was proposed using noise for initialization. However, due to performance limitations and optimization constraints, this approach compromised the usability of the distilled dataset.

• Random noise initialization is gradually being abandoned by existing distillation paradigms

  • Starting from distribution matching (DM), dataset distillation tends to use real data for initialization to achieve better aggregation of original information and higher visual realism [2, 5]. At the same time, the popular trajectory matching paradigm [3, 6], which sets a matching threshold, would fail to optimize if noise initialization were used, as the loss during the matching process would always exceed this threshold.

• Results obtained using noise initialization methods do not match the performance achieved with real sample initialization

  • We have shown the CIFAR-10 results of noise-initialized DM and PSG with $\epsilon$ approaching infinity in IPC=50. The results indicate that while these methods using noise initialization can protect the explicit privacy of the final distilled dataset, they cannot achieve the same dataset usability as KT.

  Method	TPR@0.1%FPR ($\downarrow$)	Average LPIPS Distance ($\uparrow$)	Test Accuracy ($\uparrow$)
  DM-noise	0.6±0.1	0.28	55.4±0.5
  PSG ($\epsilon\rightarrow\infty$)	1.0±0.05	0.30	58.6±0.3
  KT-DATM	0.4±0.1	0.28	69.2±0.2

• We also explain that the multiple data augmentations of KT are a form of noise addition to real data, providing privacy protection capabilities similar to low IPC at high IPC.

[5] Sun, Peng, et al. "On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[6] Guo, Ziyao, et al. "Towards lossless dataset distillation via difficulty-aligned trajectory matching." arXiv preprint arXiv:2310.05773 (2023).

Q2: In your case, does the adversary have access to the original dataset?

In line 409, we define the attacker's access, where we release the distilled dataset and the model trained on it. Therefore, in practical attack scenarios, the attacker does not possess the original dataset. Subsequently, we detail the capabilities of attackers in different experiments.

• Membership Inference Attacks
  • The attacker can only obtain the distribution of the original dataset to sample instances for training shadow models. We follow previous work [7] and use the complete set to train shadow models.
• Explicit Privacy
  • Although the attacker does not possess the original dataset, if explicit privacy is leaked, it can lead to the attacker visually capturing that the distilled dataset itself is part of the private information.
  • Explicit privacy allows the generator of the distilled data to effectively evaluate before releasing it, preventing the distilled dataset from directly leaking private information.

[7] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.

Thank you for your insightful comments on the concept of explicit privacy leakage introduced in our paper. We appreciate your feedback and would like to provide a clarification on this topic.

W1: The concept of explicit privacy leakage introduced here seems somewhat ill-defined.

We thank the reviewer for the opportunity to clarify this important concept. We will separately clarify the definition of Explicit Privacy and the motivation for proposing this concept in the dataset distillation community.

• Clarify the definition of Explicit Privacy In Definition 2 on line 207, we define Explicit Privacy as the similarity between the distilled dataset and the real private data used for training at the data level. In practice, we find that existing dataset distillation methods at high IPC directly expose the real training data used for initialization, not just visually realistic.
• Explain the motivation for defining Explicit Privacy
  • The motivation for introducing explicit privacy leakage is to address the limitations of existing data distillation methods, which claim to be privacy-preserving measures against membership inference attacks. However, state-of-the-art methods like trajectory matching often use real private training data for initialization to achieve better performance close to the original dataset. This exposure of the private data used for initialization is the key issue we aimed to address with the concept of explicit privacy leakage.
  • Furthermore, in Proposition 1 and Remark 1, we provide a theoretical analysis showing that when the IPC is high, the randomness introduced by the matching optimization stage (stage II) is not sufficient to protect the privacy of the data used for initialization, leading to the explicit privacy leakage.
  • It is important to note that differential privacy and explicit privacy are conceptually different. Differential privacy provides privacy guarantees at the random mechanism level, while explicit privacy is a data level consideration of whether the data will directly leak private information. The two concepts serve different purposes and can be complementary in a comprehensive privacy-preserving dataset distillation framework.

W2: This approach may affect the privacy of the chosen image, it should not impact the privacy guarantees for other images in the dataset. The findings in this paper appear somewhat obvious.

We appreciate the reviewer's feedback and would like to address the concerns regarding privacy leakage and the motivation behind our work.

• Privacy leakage beyond the selected images
  • In the random sampling without replacement initialization of Stage 1, the privacy information of the samples not selected is indeed protected. However, we also need to consider the complete process of dataset distillation, including the matching optimization process of Stage 2. During Stage 2, dataset distillation uses the real data initialized distilled dataset to perform matching optimization with the original private dataset. This process involves accessing aggregated information from the private data, such as gradients[1], features[2], or trajectories[3], thereby introducing privacy information of samples that were not used for initialization.
  • As shown in Figure 3, with increasing IPC, the TPR@0.1%FPR metric of membership inference attacks shows a significant increase, indicating that the distilled dataset becomes more susceptible to membership inference attacks.
• We are the first work to propose that distilled datasets directly leak initialization training information
  • Currently, dataset distillation claims applications with privacy protection, based on the random noise initialization and distribution matching paradigm [4]. Existing works often overlook the impact of changes in distillation paradigms on privacy protection. We are the first to propose that under the current distillation paradigm, distilled datasets at high IPC directly leak the privacy of initialization training samples, and we define this as explicit privacy leakage. The reason is that the randomness introduced by the matching optimization phase (line 249) decreases with increasing IPC, leading to a visual deviation of the distilled dataset from the initialized real data at low IPC (e.g., IPC=1), but directly exposing the initialized real data at high IPC (e.g., IPC=50). This has significant implications for designing privacy-preserving dataset distillation methods in the future.

[1] Zhao, Bo, and Hakan Bilen. "Dataset condensation with differentiable siamese augmentation." International Conference on Machine Learning. PMLR, 2021.

[2] Zhao, Bo, and Hakan Bilen. "Dataset condensation with distribution matching." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.

[3] Cazenavette, George, et al. "Dataset distillation by matching training trajectories." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[4] Dong, Tian, Bo Zhao, and Lingjuan Lyu. "Privacy for free: How does dataset condensation help privacy?." International Conference on Machine Learning. PMLR, 2022.

Dear Reviewer 6CFr,

Thank you once again for your insightful feedback on our submission. We would like to remind you that the discussion period is concluding. To facilitate your review, we have provided a concise summary below, outlining our responses to each of your concerns:

• Privacy leakage in data release scenarios is not guaranteed by differential privacy: Differential privacy protects membership identity, whereas explicit privacy safeguards against privacy leakage in data release (as illustrated in the new PDF, Figure 1). Using your example of DP with fixed values, if the fixed value is sensitive information, it does not violate DP, but an attacker can directly obtain this private information. The security of distilled data release is ensured through evaluations of both explicit privacy and membership inference attacks.
• The distilled dataset needs to maintain performance at a much smaller scale than the original dataset. However, the DP methods currently cannot meet this availability standard, necessitating the use of distillation techniques to ensure the utility of the concentrated data.
• A Comprehensive Revision: We have revised the title and the main text to avoid over-claiming, added scene images and descriptions to better understand the distillation data scenario, clarified the relationship with differential privacy, and reorganized the contributions of this paper.

More detailed information can be found in the general response and the revised PDF. We are grateful for your insightful comments and are eager to confirm whether our responses have adequately addressed your concerns. We look forward to any additional input you may provide.

Warm regards,

The Authors of Submission 10266.

We thank the reviewer for the comments and questions! Please find our responses to your raised questions below:

W1&W2&W3: Initial Lack of Clarity in Presentation, Undefined Terms in Theorems and Ambiguity in Figure 4.

We agree that improving the readability of our paper, especially in the early sections, would make our contributions more accessible to readers. We have made the following specific revisions to address these concerns.

• In Figure 1, we now provide explanations for all technical terms (IPC, explicit privacy leakage, MIA, and LPIPS) in the caption, with clear references to where their formal definitions appear in the paper.
• We have added explicit definitions for all variables used in Theorem 1 to ensure mathematical rigor and readability.
• We have enhanced Figure 4's caption by explaining the numerical values and clearly distinguishing between the orange and green regions for better interpretation of the results.

W4: Claim : "Free" Privacy Improvement is not substantiated.

We appreciate the reviewer's careful observation regarding our use of the term "free." We would like to clarify our statement as follows:

• The primary objective of our KT plugin is to maintain the utility of distilled datasets while providing explicit privacy protection. In the context of dataset distillation as our main focus, KT additionally offers enhanced defense against membership inference attacks.
• More specifically, regarding computational cost, we would like to highlight that the introduction of our KT plugin does not incur additional computational overhead in the distillation process.
  • Additional computational overhead of DP-generator: In contrast, existing DP-generator approaches typically require noise injection operations at each iteration step [1] and some methods [2, 3] rely on pre-training models with public data, both of which introduce substantial computational burden.
  • Comparing additional time through experiment: The KT plugin enhances the distillation samples three times during the initialization phase and averages them, completing the experiment with IPC=50 on TinyImageNet in just 3 seconds. In contrast, PSG increases the computational cost by 7 hours when $\epsilon=10$ compared to without noise.
  • Our method demonstrates superior generation efficiency by avoiding these extra computational costs while defending against membership inference attacks and explicit privacy leakage.

[1] Chen, Dingfan, Raouf Kerkouche, and Mario Fritz. "Private set generation with discriminative information." Advances in Neural Information Processing Systems 35 (2022): 14678-14690.

[2] Wang, Haichen, et al. "dp-promise: Differentially Private Diffusion Probabilistic Models for Image Synthesis." USENIX, 2024.

[3] Li, Kecen, et al. "{PrivImage}: Differentially Private Synthetic Image Generation using Diffusion Models with {Semantic-Aware} Pretraining." 33rd USENIX Security Symposium (USENIX Security 24). 2024.

W5: Claim: "Improved balanced performance with Baselines" needs a little more explanation.

We appreciate the reviewer's valuable feedback regarding the comparative analysis of privacy-utility trade-offs. We would like to clarify that the DP-generator baselines in our paper indeed support adjustable privacy budgets ($\epsilon$ values).

Our initial presentation at $\epsilon=10$ was designed to demonstrate that our KT method provides explicit privacy protection for dataset distillation while maintaining data utility. The comparison with DP-generators was intended to demonstrate that our KT plugin retains the capability to effectively distill information from the original dataset. While DP-generators provide differential privacy guarantees, they exhibit inferior downstream task accuracy with the same IPC, even when achieving comparable MIA attack success rates.

To ensure comprehensive and fair comparison, we acknowledge the importance of examining performance across different privacy budgets. Following your suggestion, we have added trade-off curves in line 460 on page 9 and the Appendix G (in line 1016 on page 19) to illustrate how DP-generators perform in terms of MIA defense and data utility under varying $\epsilon$ values. Notably, for methods like PSG, even as $\epsilon$ approaches infinity, their data utility remains inferior to KT-DATM.

Dear Reviewer opNC,

We sincerely thank you for your thoughtful reply and are pleased that our response has addressed most of your concerns, as reflected in the increased rating (8).

Your feedback about being more circumspect with terminology, particularly regarding the "free privacy" claim, remains especially valuable. We acknowledge that this aspect requires further refinement and will revise our language throughout the paper to more accurately characterize our method's benefits.

Once again, we deeply appreciate your thoughtful and constructive feedback!

Best regards,

Authors