A Reparameterized Discrete Diffusion Model for Text Generation

Thanks for your feedback and thoughtful comments! Here are our responses:

Q1: Can you take more experiments on different length language datasets that is aimed at exploring model performance boundaries? In my opinion, the diffusion model hard to generates a longer sequence than the auto-agressive language model

A1: We conduct additional experiments to explore the capabilities of RDM in open-ended text generation, particularly on Wikitext-103, which is an established dataset for language models. Please refer to the general response #3 as well as Appendix G.3 in our updated manuscript for details.

Q2: Is it simple to adapt this method to other diffusion models, such as the Gaussian diffusion model? If not, why?

A2: Thanks for your input. Our reparameterization, specifically derived for discrete diffusion models, presents challenges in direct application to Gaussian diffusion models due to their inherently different formulations (Gaussian diffusion models operate on continuous values). However, the underlying concept of separating the transition process into routing and denoising mechanisms could be beneficial in broader diffusion model classes. Exploring this reparameterization in a wider range of diffusion processes is an interesting direction for future research.

Q3: why the recursive computation for b_t is effective; I'm confused on it.

A3: $b_t$ is used to check whether the current token state $x_t$ is noiseless (i.e., equal to the ground-truth input $x_0$), which depends on $x_0$, but $x_0$ is not available during backward generation. To calculate $b_t$ without the presence of $x_0$, we propose this technique by initializing a bit vector $b_t$ at the start of backward generation. As tokens within the sequence are iteratively denoised during backward generation, this bit vector records denoised token positions and is sequentially updated to represent the noise status of each token. Therefore, the determination of $b_t$ can be accomplished independently of the direct presence of $x_0$.

Thanks for your feedback and thoughtful comments! Here are our responses:

Q1: The motivation behind the proposed method is not clearly explained, and the paper lacks a solid theoretical foundation to support the reparameterization approach. The reasoning behind dividing the diffusion of tokens into two conditions by comparing with the original input is not well-justified. Additionally, the paper does not provide a thorough theoretical analysis to support the claim that the proposed method can lead to better results compared to the original diffusion process.

A1: The developed reparameterization technique provides a strictly equivalent formulation to the vanilla discrete diffusion process. The mechanism that divides the diffusion of tokens into two conditions stems directly from this reparameterization. This theoretical grounding is elaborated upon in our paper. We further demonstrate in our experiments (especially Section 5.3 and Appendix G) that the disentanglement between the routing and denoising mechanism facilitates the separate parameterization of these two modules and makes it easier for the neural network to learn to denoise text and mitigate the risk of degeneration.

Q2: The proposed method appears to borrow heavily from masked language modeling techniques that have already been utilized in non-autoregressive text generation works, such as CMLM, DisCo, and others. Furthermore, existing discrete diffusion models like improved VQ-Diffusion have proposed advanced models that consider each token separately, making the contribution of the proposed method less novel and impactful.

A2: Masked language models can be considered as a particular instance of general discrete diffusion models, namely absorbing diffusion. Our main technical contribution involves re-formulating existing discrete diffusion processes, which decouples between routing and denoising mechanisms during backward generation. To the best of our knowledge, these results are new in the area of text generation models and have demonstrated strong performance across various tasks. Regarding ``considering each token separately'', it is a common assumption in most discrete diffusion models (if the distribution among different tokens is not factorized, it would become intractable since the size of its support would be exponential in sequence length). This assumption is commonly made to ensure the feasibility of the model.

Q3: The performance improvements demonstrated by the proposed method over the discrete diffusion baseline are not consistent across different tasks and metrics. This raises concerns about the generalizability and robustness of the proposed method for various text generation tasks.

A3: The generation quality among discrete diffusion baselines has been extensively compared in Tables 1 and 3, which shows that the reparameterized variants all outperform vanilla discrete diffusion baselines consistently by a large margin.

Q4: The choice of baselines and metrics for comparison in the experiments could be more appropriate. For example, in Figure 2 (b), the comparison with Diffuseq, a continuous baseline, is not suitable, and the performance of Diffuseq is not well-explained. Moreover, the paper does not follow the experimental settings of CMLM, which achieves higher performance in their original paper.

A4: Including continuous diffusion models like DiffuSeq in Figure 2(b) validates our method's effectiveness and efficiency against both discrete and continuous models. Regarding the performance of DiffuSeq, it often produces noisy results early on and thus leads to non-sense text with near zero BLEU score when #steps < 250, requiring substantially longer runtime to obtain decent results. In terms of the experimental settings of CMLM, our goal was to develop a model based on non-autoregressive diffusion probabilistic processes and remove any kinds of dependencies on AR approaches (unlike CMLM, whose results come from the variants distilled from autoregressive Transformer large). Hence, we opted to train the diffusion model on the original WMT14 EN-DE dataset, rather than the distilled split.

Thanks for your feedback and thoughtful comments! Here are our responses:

Q1: How does your proposed method perform on established conditional NLP datasets, such as enwik8 and text8, as previously tested in well-regarded papers (Hoogeboom et al., 2021 and Austin et al., 2021)?

A1: We conduct additional experiments to explore the capabilities of RDM in open-ended text generation, particularly on Wikitext-103, which is an established dataset for language models. Please refer to the general response #3 as well as Appendix G.3 in our updated manuscript for details.

Q2: To what degree does the quality of your text generation process rely on the choice of your training objective during the training phase? If you were to employ the loss functions utilized in Hoogeboom et al., 2021 and Austin et al., 2021 to train your diffusion model, how would this impact the quality of the generated text?

A2: We have conducted the ablation study, detailed in Figures 2a, 3a, and 3b, to systematically evaluate the effect of separate training and decoding improvements. Our results demonstrate that employing reparameterized decoding alone (i.e., applying the loss functions utilized in Hoogeboom et al., 2021 and Austin et al., 2021 to train the diffusion model, captioned as Improved Decoding), already significantly improves performance compared to vanilla baselines.

Thanks for your feedback and thoughtful comments! Here are our responses:

Q1: I always think the comparison between text diffusion models and other important non-autoregressive models is important. This is not only because of the concerns for a fair and thorough comparison. This is more because of the fact that such non-autoregressive models are text diffusion models that has a diffusion process defined as operating the discrete tokens. Levenshtein Transformer, for example, is cited yet not quite compared in many of the experiments, which limits the soundness of the experiments. I encourage the authors to add them in (many of the results can actually be directly borrowed from their original paper I think).

A1: We appreciate your insightful suggestion regarding the comparison between diffusion models and other non-autoregressive models. We agree that many such non-autoregressive models can be cast as a diffusion process, where transformations are applied to discrete tokens, and the model learns to reverse the operation, inducing a generative model. In our study, we have primarily compared RDMs against the established non-autoregressive model CMLM. This comparison highlights the superior performance of RDMs over previous non-autoregressive baselines. The Levenshtein Transformer employs a rather distinct architecture with multiple output heads dedicated to learning deletion and insertion operations, making direct comparisons challenging. However, we have included the comparison in terms of experimental results in the updated Table 1, where RDMs outperform the Levenshtein Transformer by a large margin (e.g., 27.59 vs. 25.20 BLEU on the WMT14 EN-DE benchmark).

Q2: This method is now mostly tested on semantically deterministic tasks like Machine Translation. There's still a very huge gap between getting the model to function on MT tasks and getting it to produce diverse, informative outputs on open-domain text generation tasks. With this concern, I would tend to suggest the authors to claim smaller, to restrict the claim to be a diffusion-based machine translation model instead of claiming it as a diffusion-based text generation model.

A2: We conduct additional experiments to explore the capabilities of RDM in open-ended text generation, particularly on Wikitext-103, which is an established dataset for language models. Please refer to the general response #3 as well as Appendix G.3 in our updated manuscript for details.

Q3: Is it possible to also conduct a running time study to compare RDM with other non-autoregressive models? Currently only previous diffusion sequence models and CausalLM baselines are compared. This comparison can sometimes be tricky because many irrelevant factors like the quality of implementation also impacts the running time. But I would be more convinced about the practical value if I see such results in the experiment section (since the model itself is already falling short in terms of #refinements compared to LevT, but if each refinement with RDM is computationally cheaper, there's still a chance for RDM to be faster)

A3: All models are implemented with the same codebase (FairSeq). We observe the primary factor affecting decoding runtime among non-autoregressive baselines and our diffusion models is the number of Transformer decoder calls. Consequently, the number of iteration steps (equivalent to the number of Transformer decoder calls) faithfully reflects the actual decoding speed.

For instance, the comparisons against non-autoregressive baselines are provided here (and also in Appendix G.2 of our revised manuscript): CMLM and RDM-absorbing take 44.2-45.0s and 44.0-45.3s, respectively, to decode the WMT14 EN-DE test set on one NVIDIA GeForce RTX 3090 GPU with 50 batch size and 16 iteration steps, averaged by 10 runs. Multinomial/reparameterized-multinomial diffusion models take ~46.8s; in general, the overhead, which mainly involves token-wise masking/assignment operations, is marginal compared to that of Transformer decoder calls.