Parameter-Efficient Detoxification with Contrastive Decoding

We thank the reviewer for considering our paper well-written and the model performance very good. We also appreciate the reviewer for noting that our ablation study on model size is interesting.

• By using both the generator and the detoxifier during inference, the cost is higher as compared to other methods

We respectfully disagree. In fact, cost is our approach’s strength rather than a weakness. Here we will compare our GPU memory usage and inference latency with the two top-performing baselines, namely GeDi or DExperts: (1) Our GPU memory usage is only around a third of the baselines because our detoxifier is trained with a parameter-efficient method and shares exactly the same backbone parameters as the generator (i.e., the LLM). This is different from the baselines which need to train an expert and an anti-expert LM of opposing attributes with full finetuning.

(2) Our inference latency is only two-thirds of the baselines because our detoxifier only needs to rely on the toxic part of the training data. Hence instead of running the LLM three times for each prompt during inference (one for the LLM itself, one for the expert LM, and one for the anti-expert LM), our method only runs twice because there is no anti-expert involved.

We sincerely hope that this detailed explanation alleviates the cost concern that may have negatively and greatly impacted our score.

• In Table 1, what is the perplexity for the original model (alpha = 0)?

That would be column 1 in Tables 6 and 7, where we present results with the vanilla generator (i.e., without the detoxifier).

• Previous work has applied parameter-efficient learning to detoxification

Thank you for pointing this out. Indeed Wang et al. (2022) applied parameter-efficient tuning to detoxification. We have toned down our claim in the revision. We do note a key difference though that their work assumes access to the Perspective API during training, while our work and the baselines we compare with do not, making the results not comparable to each other.

• Evaluate the framework on more benchmark datasets and downstream tasks

We selected the current evaluation benchmark because it makes it easier to compare with previous work that also focused on the same dataset. That said, we do agree that more benchmarks and downstream tasks can shed more light on the effectiveness of our approach.

We thank the reviewer for agreeing that our work leverages straightforward and efficient techniques and outperforms previous baselines.

• DetoxiGen is the same as DExperts other than operating on the probability space

We respectfully disagree. Operating on the probability space is only the difference between DetoxiGen and DExperts during inference. As summarized in the last paragraph of the Introduction section, there are two more major differences: (1) training efficiency: DetoxiGen adopts parameter-efficient training of the detoxifier while DExperts requires a full-finetuning of the language model; (2) data efficiency: DetoxiGen only requires training on the toxic data alone, where DExperts necessitates the use of non-toxic data as well.

We sincerely hope that this detailed explanation alleviates the novelty concern that may have negatively and greatly impacted our score.

• The approach overperforms previous work by a small margin

We respectfully disagree. As Reviewer FrfU and qHFk both noted, our approach achieved superior performance over baselines. In terms of Avgerage Maximum Toxicity, we brought previous state-of-the-art from 0.314 all the way down to 0.254 with the same backbone model and data. This is an even bigger gap than that between GeDi and DExperts, not to mention that the lower the toxicity baseline, the harder it is to make any improvements.

• No mention of a code release

We thank the reviewer for bringing this up. We promised code release on Page 4 Footnote 1, and we have initiated the application process to open-source this project through our institution. We have migrated this footnote to page 1 in the revision to make it more salient.

• Qualitative examples of detoxification

We thank the reviewer for the suggestion. We did not include the qualitative examples due to ethical concerns as the outputs without detoxification could seem very offensive to the readers. In our code release, we plan to provide some sample toxicity-inducing prompts and include instructions for researchers and practitioners to compare the difference in outputs.

• Effectiveness of perplexity as a measure of fluency

We agree that perplexity alone may favor models that output generic, dull continuations that lead to low perplexity. That is the motivation that our work and previous baselines add the diversity metrics as a sanity check.

• A detailed explanation of Distinct-N metrics

We thank the reviewer for the suggestion. We did not elaborate on the Distinct-N metrics due to space constraints. We have added the details in the revision.

We thank the reviewer for considering our work to have the potential to have a significant impact by introducing a straightforward and simple approach in a well-presented paper.

• Explore other parameter-efficient tuning methods such as LoRa

We went with Prompt Tuning as it is a more straightforward parameter-efficient approach that requires no change in model architecture (other than the few additional parameters from the soft token embeddings). That said, we agree that LoRa would be a nice baseline to compare with as part of the ablation studies. As the reviewer mentioned, due to space constraints, there is a trade-off on what studies to focus on — we figured that the experiments on model sizes would be very helpful for the audience to make design choices when applying our work.

We thank the reviewer for noticing that our proposed method is lightweight and valuable as it leads to superior performance over previous baselines!

• Justification of novelty

As Reviewer 9Tow mentioned: although the components of our framework (contrastive decoding, parameter-efficient tuning) are already established methods, combining and applying them toward Detoxification is novel. As discussed in the last paragraph of the Introduction section, our approach makes the data curation much easier than previous work because it only requires toxic data, and parameter-efficient tuning also makes the amount of data required much smaller than full finetuning. This improved transferability (as described in Section 1, last paragraph) is also a significant part of the novelty. We sincerely hope that this explanation alleviates the novelty concern that may have adversely impacted our score.

• Provide qualitative examples

We thank the reviewer for the suggestion. We did not include the qualitative examples due to ethical concerns as the outputs without detoxification could seem very offensive to the readers. In our code release, we plan to provide some sample toxicity-inducing prompts and include instructions for researchers and practitioners to compare the difference in outputs.

• Evaluate the framework on more benchmark datasets

We selected the current evaluation benchmark because it makes it easier to compare with previous work that also focused on the same dataset. That said, we do agree that more benchmarks and downstream tasks can shed more light on the effectiveness of our approach.