Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data

We sincerely thank you for your time and constructive feedback. We have carefully considered your Weaknesses (W) and Questions (Q) point by point and provide detailed responses below. We hope this clarification will help you see the value of our work more clearly.

W2 & Q2: On the Novelty Compared to Works [1, 2, 3]

"W: The novelty of the proposals in Sec. 3.2 is limited w.r.t. previous works. ..."

"Q: How does the proposed method meaningfully differ from prior works like [1,2,3] that also utilize domain centroids and clustering approaches?"

We are grateful to the reviewer for these valuable references. They will undoubtedly help us better position our work, and we will add all of them to our revised version.

Similar to your statement in Strengths that our work positioned in 'fine-tuning on mixed data without reliable domain labels', we wish to re-emphasize that the primary novelty of our work lies in the domain-undetermined setting and the diversity probe of LLM fine-tuning (in lines 74-79). To clarify this, we highlight the key differences with the cited works below.

Regarding [1] (Chameleon)

First, we note that [1] is contemporaneous work according to the official guidance of Call For Papers (first submitted on May 30), and it appears even after the submission (May 15). Nonetheless, we outline the distinctions here:

While both DaaR and Chameleon use domain embeddings for data reweighting, our work differs in three crucial aspects:

• Setting: DaaR operates in a domain-undetermined fine-tuning scenario where domain labels are unknown. In contrast, Chameleon's method (KRLS) use such domain information to calculate importance scores majorly in pretraining stage.
• Metric: DaaR pioneers the use of diversity to measure the utility of data, while Chameleon use domain-level importance scores derived from KRLS.
• Objective: In the context of fine-tuning, DaaR targets the enhancement of foundational abilities, whereas Chameleon focuses on specific skills like multilingual capabilities.

Regarding [2] (Aioli)

While both works investigate data mixture based on domain properties, our approaches differ fundamentally in their goals and methods:

• Objective & Setting: Aioli mainly aims to model a "mixing law" in a pre-training, domain-known setting. In contrast, DaaR is to optimize data selection for fine-tuning in a domain-undetermined context.
• Mechanism: Aioli's method relies on training dynamics (loss). DaaR is based on the intrinsic diversity of the data.
• On Embedding Distance: Aioli does not directly utilize domain centroid distance as a core mechanism, the mention of embedding distance partly in its Related Works section.

Regarding [3] (DGA)

While both methods use domain clustering, its purpose and the overall approach are distinct:

• Core Method: DGA is a pre-training method that uses gradient information. DaaR is a fine-tuning method that uses data diversity as its primary selection criterion.
• Role of Clustering: In DGA, clustering is a pre-processing step to define static, fine-grained domains. In DaaR, we utilize data synthesis to create anchor points that guide the clustering, and the clustering is an integral mechanism to train the diversity probe.

We thank the reviewer again for these valuable references. They help us clarify that DaaR’s core novelty lies not in its individual components, but in the complete system and scenario. As highlighted by Reviewers kdaW and Ge2Q in their Strength, DaaR provides a external-model-free solution for the challenging domain-undetermined setting.

W4 & Q4: On Reporting Statistical Significance

"W: No statistical significance measures, which are important given the marginal improvements."

"Q: ... can the authors report statistical significance measures to support the robustness of their claims?"

We thank the reviewer for this valuable suggestion. To bolster our claims, we provide the following points:

First, all existing results are the average of two independent, end-to-end runs (lines 112-113), and we have analyzed DaaR's stability in Appendix G.8 (lines 894-904).

To further address this concern, we conducted an additional independent run for our method, DaaR, for a total of three runs. The tables below detail the performance for each of the three seeds, followed by the calculated mean and standard deviation (μ ± σ).

• Llama3.1-8B

• Qwen2-7B

• Qwen2.5-7B

The analysis reveals that DaaR's performance is highly consistent, with the standard deviations on the final average scores being notably low (0.26, 0.08, and 0.23, respectively). This low variance confirms that our method's performance advantage over the reported baselines is robust.

We appreciate this valuable feedback and will incorporate this detailed statistical analysis into our revised manuscript.

W3 & Q3: On the Comparison with DoReMi [4] and RegMix [5]

"Q: Experiments lack domain reweighting baselines such as DoReMi [4] and RegMix [5], which could also be applied to your clustered domains."

"W: Why were ... DoReMi and RegMix excluded from the experimental comparison, ..."

We thank the reviewer for suggesting these foundational pretraining works. We already cite DoReMi (line 199) and will add RegMix to our discussion. However, we respectfully argue that including them as direct baselines would be inappropriate and could lead to an inequitable comparison:

• Divergent Settings: DoReMi and RegMix are pre-training methods designed for large-scale, iterative reweighting. Our work operates in the fine-tuning stage, which is fundamentally distinct from pretraining in its objective and data usage[1, 2, 3].
• Established Baselines: Our experimental setup aligns with the standard practice in fine-tuning data selection. To our knowledge, leading methods (e.g., Alpagasus, Instag, Deita), which we compare as baselines, do not use any pre-training method as their baselines.
• Methodological Incompatibility: Both methods require explicit domain labels, which are absent in our domain-undetermined setting. While 'could also be applied to the clustered domains', our clustering is an integral part of DaaR, designed to provide signals for our diversity probe.

[1] LIMA: Less Is More for Alignment, NeurIPS'2023

[2] What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning, ICLR'2024

[3] A Survey on Data Selection for Language Models, TMLR'2024

W1 & Q1: On the paper's clarity and structure

"W: The writing and the structure of the paper make it quite difficult to follow ... My suggestion is to restructure the paper ..."

"Q: Can the authors provide a clear algorithmic summary or flowchart ... and clarify how they interact within the overall training pipeline?"

We thank the reviewer for their valuable feedback. We will continually polish our manuscript to improve the manuscript's readability.

However, we would also like to clarify that we have already provided the following structure:

• Figure 1 already provides a schematic flowchart of the DaaR pipeline with a detaild caption. Moreover, Lines 207-210 serve as a summary that explicitly connects our DaaR to this figure.
• Our contributions in lines 53-60 are structurally aligned with Sections 3, 4, and 5 respectively.
• We also use transition paragraphs (lines 95-97, 173-178, 201-210 and 269-272) in every sections to ensure a logical flow.

We agree that the presentation can be clearer. In the revised version, we will make two key improvements:

1. Add a pseudo-code algorithm for DaaR to offer a clear, step-by-step guide.
2. Strengthen the textual mapping between our method description (Sec. 4) and the components in the Figure 1 flowchart.

Closing Remark

Thank you again for your insights and feedback! We hope these responses and the new revision can address your concerns and we would be grateful if you would consider re-evaluating your assessment of our work.

Thank you for your recognition of our method's design and comprehensive experiments. We will address your concerns regarding the Weaknesses (W) and Questions (Q) one by one.

W2 & Q2: On Potential Data Leakage

'W: Pseudo-labels for clustering incorporate downstream task samples (Sec. 4.1, Phase 1), potentially leaking test-set information...'

'Q: ... This risks contaminating pseudo-labels with test-set knowledge. Please: Report ablation results where seeds are generated without downstream task data.'

We appreciate your rigorous scrutiny of our experimental methodology. We want to assure you that our design explicitly prevents data leakage, and we welcome the opportunity to clarify this with four clarifications.

Strict Data Partitioning

First and foremost, we must emphasize that our protocol maintains strict data isolation. The few downstream samples used for seeding (2 in 5) are completely separate from the evaluation benchmark, as well as the decontamination data that used for final LLM fine-tuning (lines 642-643).

Purpose of the Injection

We posit that DaaR's performance is not dependent on these injected samples. Their role is solely to provide an initial diversity signal that accelerates the diversity augmentation process in Phase 2. Any information from the injection is heavily diluted in the subsequent large-scale seed generation and centroid computation stages.

New Ablation Study (w/o Injection)

To empirically validate this and directly address your request, we conducted a new ablation study where we removed the minor injection entirely while keeping the benchmark evaluation data same as current setting. We report on three key metrics:

• Centroid Similarity: Taking Qwen2-7B as a representative example, the domain centroids generated with and without injection are nearly identical, achieving a cosine similarity of avg over 0.987, demonstrating that the injection has a negligible impact on the domain centroid representations.

  Domains	Similarity (w/ - w/o Injection)
  Common Sense	0.9922
  Reasoning	0.9865
  Mathematics	0.9843
  Coding	0.9878

• Final Performance: The end-task performance of DaaR (w/o injection) is statistically indistinguishable from our reported results, demonstrating that DaaR is not dependent on the minor injection. The results are as follows:

  Llama3.1-8B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Inject	20.08	64.55	74.88	54.80	15.30	4.70	37.50	38.83
  DaaR w/o Inject	20.39	64.80	76.05	55.40	13.65	5.75	36.48	38.93

  Qwen2-7B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Inject	16.88	57.58	73.03	75.40	38.10	52.00	64.94	53.99
  DaaR w/o Inject	16.22	58.33	73.41	75.20	36.37	52.95	65.14	53.95

  Qwen2.5-7B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Inject	15.83	58.65	72.48	80.20	16.70	64.20	68.29	53.76
  DaaR w/o Inject	14.91	58.32	72.55	79.70	16.30	63.70	70.65	53.73

• Generation Efficiency: However, the diversity augmentation process became significantly less efficient. Without the initial seed diversity, it required approximately 4 times more generation attempts to meet the same diversity threshold (as in Table 14), confirming its role as an accelerator.

  Domains	Attempts (w/ Injection)	Attempts (w/o Injection)
  Common Sense	31	134
  Reasoning	61	248
  Mathematics	236	819
  Coding	21	105

Context in Related Literature

This practice of using a small, targeted set to guide a broader data selection process is a well-established strategy in related work, such as gradient-based influential selection LESS [1] and targeted-distribution importance resampling DSIR [2]. While our method differs, the principle of a minor, guided injection to improve efficiency will not lead to the leakage.

We thank you again for raising this critical point. We will add this new, detailed ablation study to our revised manuscript, which we believe will greatly enhance its rigor and transparency.

[1] LESS: Selecting Influential Data for Targeted Instruction Tuning, ICML'2024

[2] Data Selection for Language Models via Importance Resampling, NeurIPS'2023

W1 & Q1: On the Formalization of the Theoretical Underpinning

'W: The core claim—mutual information maximization underpins DAAR—lacks explicit formalization or proof (mentioned in Sec. 3.4 but not elaborated).'

'Q: Sec. 3.4 states DAAR’s strategy "must simultaneously maximize mutual information," but no proof connects Eq. 5–6 to DAAR’s mechanics. Please formalize ... is maximized. And clarify if entropy prediction directly optimizes...'

We are very grateful for this deep engagement with our theoretical framework, and we appreciate the opportunity to clarify our reasoning and correct an imprecision in our manuscript.

First, we wish to clarify the intended role of our theoretical analysis (lines 173-178). Its primary purpose is to provide a principled perspective that explains our empirical observations in Section 3 and offers a strong motivation for the design of DaaR in Section 4.

Regarding "Mutual Information Maximization"

We acknowledge that our use of the term "mutual information maximization" in the introduction section was an imprecise statement. Our initial intention was to use this term to conceptually capture the goal of enriching sample-level diversity. However, we recognize this is not a formal maximization of I(X;C) in information theory and thank you for pointing it out. We will immediately correct this terminology throughout the revised manuscript.

Instead, the theoretical foundation of our work is rooted in Importance Sampling, as detailed in Section 3.4 and derived in Appendix E. This framework allows us to analyze how data diversity impacts the optimality of data selection.

Clarifying the Link Between Theoretical Insight and Entropy-based Method (Eq. 9)

The connection between our theory and DaaR's entropy-based selection is best formalized as a principled heuristic, which we present in the following remark:

• Remark: Our analysis in Sec. 3.4 shows that the importance weight w(x) = q(x)/p(x) for data selection is suboptimal when the domain assignment is deterministic, i.e., p(c=k*|x) = 1 for some domain k*. This condition is equivalent to the conditional entropy H(C|X=x) = 0. Since directly optimizing for the ideal q(x) is intractable in our domain-undetermined setting, DaaR adopts a theory-guided heuristic to select samples x that are maximally distant from this known suboptimal condition. We use H(C|X=x) as a direct proxy for this distance. Therefore, the selection objective of DaaR is to argmax_x H(C|X=x), which is implemented by selecting samples with high predicted entropy ψ_div(x) ≈ H(C|X=x).

This remark connects our theoretical insight—that zero conditional entropy is suboptimal—to our practical method of rewarding high predicted entropy.

Thank you once again for your invaluable feedback. We will take the following actions in our revision:

1. We will remove all imprecise mentions of "mutual information maximization" and accurately describe our theoretical basis as Importance Sampling.
2. We will incorporate the formal discussion and remark above to explicitly bridge the gap between our theoretical insights and the design of our method.

Closing Remarks

Thank you once again for your valuable feedback, we have endeavored to address each of your concerns thoroughly in our responses and planned revisions.

We hope these clarifications have resolved your initial concerns, we would be very grateful if you would consider reconsidering your assessment of our paper. We look forward to any further discussion and welcome any additional questions you may have.

Dear Authors,

First, the authors experimentally demonstrate that not using downstream task data as seed data can achieve comparable results, albeit potentially increasing clustering costs. These experiments are well-conceived and effectively address my concerns on this point.

Regarding the mutual information question, the authors decided to remove the imprecise mentions of "mutual information maximization" and accurately describe their theoretical basis as Importance Sampling, stating: “We will remove all imprecise mentions of "mutual information maximization" and accurately describe our theoretical basis as Importance Sampling.” I consider this a rigorous and responsible approach. Having “temporarily set aside” the influence of the mutual information concept, I carefully re-examined the manuscript and the authors' response, leading to three new questions:

In the proposed work, a classifier $\psi_{dom}$ with an LLM backbone is first used to predict pseudo-labels, thereby obtaining predictions $p(c(x)=k|x)$, and then the predictive entropy of this classifier is calculated. Subsequently, the authors train a separate regression predictor $\psi_{div}$ using the same backbone. I don't understand why $\psi_{dom}$ isn't used directly to compute the domain probability distribution of the training data and then apply the entropy calculation formula (Equation 9) to obtain the sample entropy. Adding an extra model training step seems unnecessary.

Based on the authors' response, the logic for performance gain stems from the statement: “importance weight $w(x) = q(x)/p(x)$ for data selection is suboptimal when the domain assignment is deterministic”, thus inferring that “DaaR adopts a theory-guided heuristic to select samples x that are maximally distant from this known suboptimal condition.” The essence of this logic appears to be: since A is known not to be optimal, maximizing the distance from A should lead to something better. This reasoning doesn't seem entirely robust to me. My concern would be alleviated if the authors could propose a more solid theoretical explanation for the foundation of this work.

In my understanding, $M(x)$ can be viewed as embeddings of LLM-generated training samples. Using k-means in this embedding space yields semantic cluster centroids that integrate the training data distribution with the LLM's intrinsic features. These centroids are then used as labels to train the classifier, after which samples with maximum entropy are sought. This process can be interpreted as seeking samples far from the k-means centroids, situated at the "edges" of clusters, thereby increasing training data diversity. This aligns with the authors' statement in the last paragraph of Section 4.2 "Data Selection": “Building on the theoretical insights in Sec. 3.4, data points that are closer to other centroids and more dispersed within their own centroid are more beneficial for enhancing the comprehensive capabilities of the model.” My question is: could this method produce an adverse extreme effect, i.e., excessively focusing on marginal data while neglecting data near the semantic cluster centroids (which are typically considered most representative of the cluster)? Perhaps a sampling method that balances both marginal and central data would yield better results?

Regardless, the methodology employed in this work holds significant practical value, and the experimental section is thorough. I will consider raising my score and look forward to the authors' response to my new questions, which I will consider for a further increase in my score.

The author's response addressed my concerns and I will consider raising my score.

Thanks for your valuable feedback! We are deeply encouraged by your recognition of our work's novelty and comprehensive evaluation. We also greatly appreciate your insightful suggestions, we will address each of your Weaknesses (W) and Question (Q) below.

W1: On the Compounding Errors and Robustness of DaaR

'Complex framework with potential for compounding errors: The framework is multi-staged and complex. ... An error or instability in any of the early stages, such as the initial centroid generation, could propagate through the entire system ...'

This is a very insightful point. You are correct that since DaaR operates as a multi-stage pipeline, a thorough analysis of its stability is crucial. We did touch upon this consideration in the main text (lines 266-267) and provided an initial stability analysis in Appendix G.8 (lines 895-904).

In summary, our existing analysis suggests that the key components of DaaR exhibit robustness: the LLM-based seed generation is consistent, the k-means clustering is stabilized by deterministic initialization, the reward probe training converges reliably, and the final data selection strategy has built-in redundancy.

However, your observation that we should more explicitly analyze the potential for compounding errors is well-taken. To thoroughly address this concern and directly validate the robustness of our framework against error propagation, we have conducted the following new analyses based on three fully independent experimental runs.

• [Experiment Setup]

We performed a third independent run, supplementing the current two independent runs mentioned in lines 112-113. Each run is an end-to-end execution of our pipeline. We present results for Qwen2-7B as a representative case; other models showed similar trends and will be added to the paper.

• [Observation 1]: Robustness of Centroid Generation.

To evaluate the stability of the critical initial stage, we computed the pairwise cosine similarities of the final centroid embeddings from the three runs.

The centroid embeddings are remarkably similar (all >0.98), which demonstrate the stability of our initial centroid generation, showing it is not a significant source of variance, especially compared to the value in Figure 11 and Table 11.

• [Observation 2]: Stability of Final Data Selection.

We then examined if initial variations propagate to the final data selection by calculating the data overlap for the top 20% (8000) of selected samples across the three runs.

The results show an significantly high overlap, with an average of 96.4%, confirming that our data selection process is robust and largely unaffected by initial variations.

• [Observation 3]: Consistency of Final Performance.

Finally, the stability of the entire pipeline is reflected in the end-task benchmark performance.

The final results are highly consistent, culminating in an average score with a very small standard deviation of just 0.08. This end-to-end stability provides strong empirical evidence that our framework is robust. (Complete results can be found in our reply to W4 & Q4 of Reviewer Bbh9).

In summary, and following your valuable advice, we will make two key revisions to the paper:

1. We will integrate a more prominent discussion of the framework's stability, currently in the appendix, into the main body of the paper.
2. We will incorporate the new ablation experiments described above into our ablation studies (Section 5.3) to explicitly address the concern of compounding errors.

W2: On the Limited Scope of Models

'Limited scope of models: The experiments are primarily conducted on models within the 7-8B parameter range. It remains unclear .. larger models (for example, 70B or more) or different model architectures.'

Thank you for your constructive suggestion! We agree that demonstrating DaaR's effectiveness across a wider range of model architectures and scales would significantly strengthen our claims.

Our current experiments already span three distinct architectural families: the widely-used Llama and Qwen-2.x architecture, as well as the latest SOTA Qwen-3 architecture. Across all these, DaaR has consistently demonstrated its effectiveness.

However, we acknowledge that our original study did not include models of different sizes. Given the considerable time and computational resources required to run our full suite of experiments (10 diversity-controlled settings and 12 baselines with 2 independent runs), we conducted a more targeted set of experiments on two additional models: Llama3.2-3B and Qwen2.5-14B.

For these experiments, we compared DaaR against two key baselines: Raw and Random (a strong baseline in our existing results). Our supplementary results are as follows:

• On Qwen2.5-14B (a larger model)

• On LLaMA3.1-8B (a smaller model):

As the new results demonstrate, DaaR consistently outperforms on both the larger Qwen2.5-14B and the smaller Llama3.2-3B. On Qwen2.5-14B, DaaR achieves the highest average score (61.54), marking an improvement over the strong Random baseline with 0.74. Similarly, on Llama3.2-3B, DaaR (29.64) maintains its advantage.

These results provide an evidence that our method is effective across both LLM architectures and model scales. We will incorporate these findings as an additional ablation study in our revised manuscript.

Q1: Conjecture on the Performance Variance on MMLU

'The OOD evaluation on MMLU shows that DAAR achieved the top score on Qwen2-7B but was outperformed by other baselines on Qwen2.5-7B. Do the authors have a hypothesis ...?'

That is an excellent observation, it is indeed interesting that DaaR places second on Qwen2.5-7B. While a definitive cause warrants deeper investigation, we offer two primary hypotheses based on our analysis:

• Domain Alignment vs. Diminishing Gains: Our detailed results in Table 6 shows DaaR's strength on Qwen2-7B is largely driven by STEM subjects, which align with the core domains (math, code, etc.) our method prioritizes. On the more capable Qwen2.5-7B, DaaR still effectively boosts STEM performance. However, for an already strong LLM, the marginal gains on less-related domains (e.g., Humanities) may diminish, leading to a slightly lower overall average compared to a method like SuperFilter that might have a different selection bias.
• LLM Sensitivity: The weaker base model (Qwen2-7B) appears more sensitive to data selection, exhibiting larger performance variance across all methods. Conversely, on the stronger Qwen2.5-7B, the performance of all baselines is more stable and converges. In this less volatile regime, the strong, diversity-driven focus of DaaR on STEM might introduce a slight trade-off in other areas, resulting in a marginal difference.

We believe the OOD setting is a fascinating area for future work and would welcome any further discussion.

Closing Remarks

Once again, we sincerely thank you for your recognition and invaluable feedback. We hope our responses have thoroughly addressed your concerns regarding robustness and model scale.

We kindly hope that these clarifications and additions might further strengthen your confidence in our work, your support is immensely encouraging to us. If you have any additional concerns or queries, we warmly invite you to share them with us.

Thank you for your recognition of our novelty and empirical rigor. We will address your concerns regarding the Weaknesses (W) one by one.

W3: On the 'True' Domain Distributions

'Lacks grounding in true domain distributions:The diversity reward signal ... depends on noisy pseudo-labels ... If (ψdom) misclassifies domains, entropy becomes a meaningless proxy, propagating error to (ψdiv) and data selection.'

Thank you for this insightful question about the stability of our method and its reliance on pseudo-labels.

First, as mentioned in your Strength 1, we wish to emphasize the core design principle and the setting of DaaR: it is a 'closed-loop, model-aware system designed to eliminate dependency on external domain label' (from SOTA embedding model or human annotation). Our goal is not to replicate human annotations, but to understand the data's structure from the LLM's own perspective (lines 239-240).

To directly address your concern about "noisy" pseudo-labels versus "true" labels, we conducted a new ablation study as follows. We replaced our model-aware pseudo-labels from clustering with the ground-truth (GT) human-annotated labels (which is only possible in our controlled setting, not in DaaR's target domain-undetermined scenario) and reran the entire pipeline. We report on three key metrics:

• Probe Training Convergence: Using ground-truth labels, the domain predictor also converged successfully, achieving the test accuracy of 92.7%, confirming that both labeling schemes can allow the probe to distinct domains.

• Difference in Final Data Selection: The table below shows the overlap in the final selected data. The divergence in the selected data confirms that pseudo-labels and ground-truth labels guide the prioritization process toward distinct data subsets.

  Overlap	Pseudo vs. GT
  Llama3.1-8B	87.3%
  Qwen2-7B	83.1%
  Qwen2.5-7B	84.9%

• Final Performance: The most critical comparison is the end-task performance:

  Llama3.1-8B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Pseudo	20.08	64.55	74.88	54.80	15.30	4.70	37.50	38.83
  DaaR w/ GT	22.54	65.52	73.43	53.80	14.35	4.40	36.50	38.65
  Diff	+2.46	+0.97	-1.45	-1.00	-0.95	-0.30	-1.00	-0.18

  Qwen2-7B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Pseudo	16.88	57.58	73.03	75.40	38.10	52.00	64.94	53.99
  DaaR w/ GT	18.75	58.35	72.02	73.40	35.75	51.50	64.01	53.40
  Diff	+1.87	+0.77	-1.01	-2.00	-2.35	-0.50	-0.93	-0.59

  Qwen2.5-7B	nq	triviaqa	hellaswag	gsm8k	math	mbpp	humaneval	Avg
  DaaR w/ Pseudo	15.83	58.65	72.48	80.20	16.70	64.20	68.29	53.76
  DaaR w/ GT	16.04	58.75	71.87	79.40	16.30	64.25	67.76	53.48
  Diff	+0.21	+0.10	-0.61	-0.80	-0.40	+0.05	-0.53	-0.28

This experiment reveals two critical insights:

1. Model-aware pseudo-labels are more effective. Using "true" labels leads to an average performance drop compared to our standard DaaR. This suggests that an LLM's internal understanding of data can be a better guide for fine-tuning than human-defined domain label.
2. "True" labels can introduce suboptimal biases. For instance, we observed that using human labels biased the final model's capabilities, often over-strengthening certain domains (e.g., common sense) at the expense of others, leading to lower performance on others. This suggests apparent 'misclassifications' caused by pseudo-labels may actually more functionally effective groupings from the LLM's perspective.

In addition to this new experiment, our existing analysis in Appendix G.8 already demonstrates the component-level stability of DaaR (e.g., consistent seed generation, deterministic clustering, and reliable probe convergence).

We greatly appreciate this constructive point. We will add this new experiment to our ablation studies, as it strongly highlights the advantages of DaaR's model-aware design.

W1: On the Embedding Representation

'unvalidated embedding representation: Assumes embedding-layer features accurately reflect data distribution (sec 3.2) without regression/ablation experiments to verify fidelity to true semantic distribution.'

Thank you for this excellent question, which naturally extends the discussion from our reply to W3 above.

Our reliance on the model's own embedding layer is a deliberate design choice, rooted in DaaR's model-aware and external-model-free philosophy. As our new ablation experiment in W3 demonstrated, imposing an external "true" perspective (like human labels) can actually misguide the fine-tuning process.

The reason of selecting embedding layer is in lines 569-573. And we posit that the principle applies here: a more relevant semantic distribution for fine-tuning an LLM is the one perceived by the LLM itself. Besides, defining a universal "ground-truth" or a SOTA semantic distribution is a complex challenge in recent works [1,2] that lies beyond the scope of this work.

Nevertheless, to investigate the practical implications, we processed our full dataset (40K) using several SOTA external embedding models of a similar model size (7B). To ensure a fair comparison, all experimental setups were identical, utilizing a batch size of 16 and the Transformers library for embedding extraction. This analysis revealed two key points:

• No Single "True" Representation Required: While each embedding model generates a distinct semantic space, all of them successfully differentiate the domains (since the new picture can't be displayed, it is similar as Figure 12).
• Significant Computational Overhead: Using external models introduced substantial computational costs. Specifically, their processing times were 175 and 192 times longer, respectively, and they even led to OOM errors in 40GB of GPU. This highlights the significant efficiency advantage of utilizing the model's native embedding layer.

Given these findings, both the alignment with our model-aware principle and the significant computational overhead, we posit that using the LLM's own embedding layer is the sound and efficient choice for DaaR. We will add a note on this to the revised manuscript.

[1] Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models

[2] Jasper and Stella: Distillation of SOTA Embedding Models

W2: On the Phrasing of Performance Gains

'DAAR shows minimal improvement (0.1%) for Llama3.1-8B over the best baseline (SuperFilter) in Table 2, undermining claims of "notable" superiority.'

Thank you for this observation. Your feedback is very helpful for improving the precision of our manuscript.

First, we wish to clarify that the experimental task is a particularly challenging task (as detailed in lines 285-291), where most existing baselines show limited effectiveness. Our primary claim is that DaaR achieves consistent SOTA performance across all tested models and scales.

To further validate this consistency across a broader range of setups, we have conducted supplementary experiments on two additional models, Llama3.2-3B and Qwen2.5-14B. For these experiments, we compared DaaR against two key baselines: Raw and Random (a strong baseline in our existing results). Our supplementary results are as follows:

• On Qwen2.5-14B (a larger model)

• On LLaMA3.1-8B (a smaller model):

As the new results demonstrate, DaaR consistently outperforms on both the larger Qwen2.5-14B and the smaller Llama3.2-3B. On Qwen2.5-14B, DaaR achieves the highest average score (61.54), marking an improvement over the strong Random baseline with 0.74. Similarly, on Llama3.2-3B, DaaR (29.64) maintains its advantage, that are consistent with our prior findings on the Llama architecture.

Nevertheless, we agree that the term "notable" may not be the most precise descriptor for every individual result. Following your valuable suggestion, we will revise our manuscript to use more accurate language, such as "consistently" to characterize the overall performance pattern.

Closing Remarks

Thank you again for your insights and feedback! We hope these responses and the new revision can address your concerns and enhance your confidence in the acceptance of this paper. If you have any additional concerns or queries, we warmly invite you to share them with us.