W1: I strongly encourage the authors to include at least a minimal discussion of related works in the main text. I encourage the authors to clarify how their method differs from prior work and to clearly highlight which components are original.

R1: Clarify our difference from previous work. Thank you for the constructive suggestion. To help readers better understand how our method differs from prior methods, we will include a discussion of related work in the main paper:

Existing defense methods can be broadly categorized based on when they are applied: during the alignment stage or the fine-tuning stage. Alignment-stage defenses aim to preemptively enhance model robustness against simulated attacks, by improving resistance to harmful perturbations [1,2] or actively forgetting harmful knowledge [3]. Fine-tuning-stage defenses focus on reducing the impact of harmful data during the fine-tuning, such as by increasing the proportion of safe samples [4] or filtering out potentially harmful samples [7].

The method proposed in this paper should be classified into a fine-tuning-stage solution. However, it differs fundamentally from prior work in that we propose a novel loss-based data scheduling mechanism derived from Bayesian inference principles, whereas previous approaches typically perform hard-label data selection based on less effective heuristic rules [5,6] or manually tuned thresholds [7]. Moreover, unlike the current SOTA method Booster [1], our method does not require attack simulation, making it more practical and efficient, while also demonstrating strong performance and adaptability.

[1] Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation. ICLR 2025.

[2] Vaccine: Perturbation-aware alignment for large language models against harmful fine-tuning attack. NeurIPS 2024.

[3] Representation noising: A defence mechanism against harmful finetuning. NeurIPS 2024.

[4] Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack. NeurIPS 2024.

[5] A Survey on LLM-as-a-Judge. arXiv 2024.

[6] Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts. arXiv 2025.

[7] Safety-aware fine-tuning of large language models. NeurIPS workshop 2024.

W2: Concerns regarding the computational cost of BDS in realistic setting.

R2: Thank you for the thoughtful suggestion. We would like to clarify the efficiency of our method BDS by emphasizing the following three points:

(i) Efficiency of BDS compared with to vanilla fine-tuning. We would like to clarify that BDS does not introduce significant overhead on top of vanilla fine-tuning, which serves as the lowest possible baseline cost in the fine-tuning-as-a-service scenario. This is because BDS only additionally maintains a set of sample weights, which are updated in a lightweight manner using simple additive operations on the loss (see Eq. (16)).

(ii) Efficiency when scaling to larger fine-tuning datasets. BDS remains efficient even when scaling fine-tuning datasets, since the number of weights scales linearly with the number of samples, and updating them requires minimal computation compared to model optimization. Moreover, our proposed neural scheduler further addresses the scalability issue by decoupling the number of trainable parameters from the dataset size, ensuring that the optimization complexity does not increase as the dataset grows.

(ii) Significant efficiency gains over existing SOTA defense. Compared to the SOTA defense method Booster, BDS is over 3× faster and consumes less than half the GPU memory. This efficiency gain comes from the fact that Booster relies on bi-level adversarial training, which involves expensive second-order gradient approximations, while BDS requires only the standard backward pass.

Empirical Support. To better illustrate the efficiency of BDS, we will supplement the theoretical complexity analysis in Appendix G.1 with practical runtime and memory benchmarks in the final version:

As shown in the table above, under the same training setting, (i) BDS introduces no significant overhead compared to vanilla fine-tuning, which serves as the lowest possible baseline cost in the fine-tuning-as-a-service scenario. (ii) BDS achieves substantial improvements in training speed and memory efficiency over SOTA baseline Booster.

Thanks for the authors' detailed response. It addressed all the concerns I have. Please also include these discussions of related works in the final version. I will raise my score to 5. Good work!

W1: The iterative posterior sampling process of BDS would introduce additional computational overhead.

R1: (i) Efficiency of posterior sampling in BDS. Thank you for your thoughtful comments. The iterative posterior sampling process in BDS, defined by the update rule $\theta_{t+1} = \theta_t - \eta \cdot \nabla L(\theta_t) + \sqrt{2\eta} \cdot \epsilon_t$, can be interpreted as standard gradient descent augmented with a Gaussian noise term. Crucially, the injection of noise incurs negligible additional computational overhead compared to the dominant cost of gradient computation. Therefore, the overall computational overhead of BDS remains comparable to standard backpropagation, without introducing significant additional cost.

(ii) Empirical support. To further demonstrate the efficiency of BDS, we supplement the theoretical complexity analysis in Appendix G.7 with a practical benchmarks of time and memory cost:

As shown in the table above, our method (BDS) is over 3× faster than the SOTA method Booster, while consuming less than half the GPU memory. This efficiency gain is due to the fact that Booster relies on bi-level adversarial training, which involves expensive second-order gradient approximations, while BDS requires only the standard backward pass.

W2: Have the authors considered comparing BDS against a pre-trained soft safety classifier of input data?

R2: Comparison with pre-trained safety classifier. Thanks for your constructive suggestion. We include an additional baseline: Llama Guard 2, a pre-trained safety classifier fine-tuned from LLaMA-3 using the 13K Anthropic dataset. The updated comparison is shown below:

As shown in the table, BDS significantly outperforms Llama Guard 2 in both fine-tuning accuracy and harmfulness score. We further observe that, among 1000 harmful samples, Llama Guard 2 exhibits a high leakage rate of 37.5%, i.e., it misclassifies 37.5% of harmful data as safe. This indicates that relying solely on pre-trained safety classifiers may introduce notable risks during data filtering, whereas our method offers more reliable data weighting and enhanced safety.

W1: Concerns regarding efficiency analysis.

(i) Add a table comparing training time, memory and GPU overhead, and FLOPs with standard optimizers (e.g., Adam, SGHMC).

(ii) Supplement the time complexity analysis in Appendix G.1 with concrete benchmarks.

R1: (i) Compare SGLD with standard optimizers. We include the comparison table below to clarify the resource usage of different optimizers and samplers. While these methods differ in their update rules, they do not lead to significant differences in overall training overhead, as the optimizer step typically accounts for only a small fraction of total training overhead compared to the gradient computation $\nabla L(\theta_t)$.

(ii) Practical time and memory analysis of our method. In addition to the theoretical complexity analysis provided in Appendix G.1, we will include a practical runtime and memory benchmark in the final version. As shown in the table below, our method (BDS) is over 3× faster than the SOTA method Booster, while consuming less than half the GPU memory. This efficiency gain is due to the fact that Booster relies on bi-level adversarial training, which involves expensive second-order gradient approximations, while BDS requires only the standard backward pass.

W2: Concerns regarding reliance on high-quality alignment dataset.

(i) Consider adding a performance analysis for low-resource $D_{\rm safe}$ settings.

(ii) Discuss whether a “safe attribute migration” policy could help amortized schedulers generalize with minimally aligned data.

R2: Reliance on alignment data ($\mathcal{D}_{\text{safe}}$). We acknowledge that our method relies on an alignment dataset $\mathcal{D}_{\text{safe}}$. However, this reliance can be mitigated in several ways:

(i) Robustness to small alignment datasets. Our method remains highly effective even with very limited alignment data. As shown in Table 6 of our main paper (also the table below), BDS consistently achieves harmful scores < 2 with as few as 100 alignment samples, outperforming the SOTA baseline Booster by over 60%, which shows the feasibility of our method in low-resource conditions:

(ii) Generalization via “safe attribute migration”. Our method enables a practical strategy to reduce reliance on domain-specific $\mathcal{D}_{\rm safe}$: leveraging alignment data from a data-rich domain as a surrogate for data-scarce domains.

As shown in Appendix D.3 (Table 8), a scheduler trained on alignment data from a specific domain (e.g., BeaverTails) generalizes well to cross-domain attacks (e.g., RealToxicityPrompts and AdvBench). This demonstrates the feasibility of using alignment data from a data-rich domain as a surrogate for data-scarce domains, thus alleviating the reliance on domain-specific $\mathcal{D}_{\rm safe}$.

W3: Concerns regarding the discrepancies between the attacker’s and defender’s models or alignment objectives.

R3: Thank you for your thoughtful comments. We would like to first clarify the threat model of harmful fine-tuning as defined in Section 2 of the main paper. In our setting, the attacker does not use or access any model. Instead, the attacker first uploads harmful fine-tuning data to a server. Then, the server performs fine-tuning with an LLM. Finally, the trained LLM is returned to users via an API. This setup reflects realistic deployment scenarios of fine-tuning-as-a-service, as offered by mainstream LLM providers (e.g., OpenAI, Mistral).

As a result, since the attacker does not use or access the model being fine-tuned, there is no discrepancy between the attacker’s and defender’s models in our setting. Therefore, concerns regarding discrepancies in LLM architecture do not apply to this specific threat model.

However, we appreciate your broader perspective. In more complex attack scenarios, an attacker may use a heterogeneous model to optimize harmful examples before uploading them. While this goes beyond the assumptions of our current threat model, we agree it is a valuable direction for future work. Specifically, future work could investigate how well the defense generalizes to such cross-model adaptive attacks, where the attacker uses heterogeneous models to craft more stealthy harmful data. We will include this discussion in the final version.

W1: The figure 1 requires clearer explanation to help reader better understand the density and weights concept as the introduction figure in the paper.

R1: Thank you for your suggestion. In Fig.1, the term "density" in the largest panel was intended to refer to the density of data weights. To avoid confusion, we will replace the term "density" with "weight" and use consistent terminology throughout the paper.

W2: Some important information for understanding the methods, such as experiments details, should be directly included into the paper rather than in the appendix.

R2: Thank you for your suggestion. We will move the experimental details currently in Appendix B.3 and B.4 into the main paper to improve clarity and completeness.

W3: The method is still based on a datset with safe examples (harmful questions with safe answers). The model may perform not well on unexpected attacks with harmfulness beyond the alignment dataset.

R3: (i) Attack beyond the alignment dataset. Thank you for raising this concern. We address it in Table 8 of our main paper, which presents results where the alignment dataset and the attack dataset differ significantly. The table below summarizes these datasets, highlighting their differences in format and domain:

(ii) Empirical support. As shown in Table 8 of our main paper, even when the alignment and attack datasets differ significantly in both format and domain, our method consistently maintains a harmfulness score below 3 and significantly outperforms existing SOTA baselines.

W1: Concerns regarding the generality of BDS method beyond safety.

R1: Thank you for your insightful comment. We agree that our method has broader applicability as an adaptive data scheduling framework for enforcing a variety of constraints during training. We will include the following discussion in the final version. The table below illustrates how the proposed BDS method can be adapted to different scenarios by appropriately selecting $D_{\rm ft}$ and $D_{\rm constraint}$.

Empirical results. The table below shows that the BDS method can be effectively applied across various scenarios. The baseline "Mix" refers to simply combining $D_{\rm ft}$ and $D_{\rm constraint}$ as the training set.

As shown in the table, BDS can be flexibly adapted to diverse constrained training scenarios through appropriate dataset design, consistently outperforming the simple mixing baseline across all scenarios. We will add a dedicated discussion on the generality and extensibility of the BDS framework in the final version.

W2: Lack of discussion on adaptive attack and potential mitigation strategies.

R2: Thank you for your constructive suggestion. We will include a discussion on adaptive attacks and potential mitigation strategies in the final version.

(i) Adaptive attack analysis. We explore two adaptive attack strategies under the assumption that the attacker has knowledge of the defense scheduler:

1. Selection-based adaptive attack: The attacker constructs a proxy scheduler and selects harmful samples with highest weights to poison $D_{\rm ft}$.
2. Optimization-based adaptive attack: The attacker constructs a proxy scheduler and directly optimizes the discrete harmful inputs to maximize the scheduler output (i.e., $\max_{\mathbf{z}} \boldsymbol{\phi}(\mathbf{z})$). Since $\mathbf{z}$​ lies in the discrete token space, discrete data optimization methods such as GCG optimizer [1] can be used.

(ii) Empirical results of adaptive attack. The table below demonstrates the performance of the adaptive attacks. We assume that the candidate attack data includes unseen harmful data from BeaverTails, RealToxicityPrompts, and AdvBench.

1. For the selection-based adaptive attack, we observe limited improvement over non-adaptive attacks, as harmful samples generally receive much lower weights than benign ones (see Fig. 6 in Appendix, page 19), making it difficult to collect naturally stealthy attack data. This also suggests the need for more sophisticated crafted attack data to effectively bypass the scheduler.
2. For the optimization-based adaptive attack, the effectiveness is constrained due to the trade-off between harmfulness and stealthiness [2]: optimizing for higher weights (i.e., stealthiness) could reduce the sample's harmfulness. In addition, discrete optimization on long text sequences is often unstable and less effective [1].

(iii) Potential defense against adaptive attack. To defend against adaptive attacks, future work could explore how to construct "worst-case stealthy" harmful data $\hat{\mathcal{D}}_{\rm harmful}$ that effectively balances the harmfulness-stealthiness tradeoff, and how to use it to adversarially train the scheduler.

[1] Universal and Transferable Adversarial Attacks on Aligned Language Models.

[2] Information-Theoretical Principled Trade-off between Jailbreakability and Stealthiness on Vision Language Models.

W3: Concerns regarding mathematical presentation for non-Bayesian readers.

R3: Thank you for your advice. In the final version, we will thoroughly re-organize the mathematical presentation and include more preliminaries and references to improve accessibility for non-Bayesian readers.

W4: It might be more fair to move RepNoise to D.10 if it was used in an offline fashion.

R4: We agree that RepNoise is an offline method and will move it to the appendix accordingly for better clarity.

W5: I recommend the authors report the entire dataset (Refusal-Safe, Partial Refusal-Safe, Answer-Safe, Answer-Unsafe, Partial Answer-Unsafe, Refusal-Unsafe).

R5: Thank you for your thoughtful suggestion. We now report the full evaluation results on XSTest for the aligned model. The results are summarized below:

As shown in the table above:

(i) We observe a number of refusals to safe prompts, which we attribute to the base model’s inherent limitations in language understanding [3]. An example is shown in the table below (first row).

(ii) We also observe a small number of answers to unsafe prompts, where these responses are not classified as refusals. Example are provided below (second row).

[3] XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. NAACL 2024.

W6: Concerns regarding the claimed infeasibility of attack simulations.

R6: Thank you for your constructive comment. We agree with your comment that attack simulations are not inherently infeasible, as defenders can indeed simulate attacks within a pre-defined and limited threat model. In the final version, we will revise our claim to be more nuanced:

Existing defense strategies preemptively build robustness via attack simulation but suffer from fundamental limitations: (i) the infeasibility of performing attack simulation beyond bounded threat models, due to the defender's limited ability to anticipate unknown attacks; and (ii) limited adaptability to varying attack settings, as simulation fails to capture their variability and complexity.

This revision reflects your insight that simulation is feasible under constrained conditions, while preserving our original motivation—that the space of possible attacks are too diverse and unpredictable to be fully captured by simulation alone.

W7: Insufficient discussion of negative societal impact about viewpoint suppression.

R7: Thank you for your insightful comment. This is an important point that we had not fully considered, and we appreciate your perspective. We will include a discussion in the final version on the potential misuse of our method to enforce specific ideological positions or suppress diverse perspectives in LLMs.