Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning

We are deeply impressed by the reviewer’s profound knowledge and insightful comments. Addressing these questions was an enjoyable experience. All discussions will be incorporated into the camera-ready version.

---

[Q-1] Why is the preference for high priors and repetition problematic, and what does R2FT aim to achieve? (Regarding your Question1, 3)

= First, let me briefly respond to the statistical questions:

$ Average length of response in training data:

57.47 (GPT-2 tokenizer)

$ Does a single instance in the training data consist of multiple question–answer pairs?

No. The behavior (“Question” and “Answer” appearing on top candidates) is not directly learned.

$ Do the probability patterns for repeating and high-prior tokens in the training data resemble those in Figure 3?

In the training dataset, the probability of high-prior and repetitive tokens (we will note them “easy tokens”) resembles the pattern of the model, declining after an initial peak. However, the peak occurs much earlier (within 1~2 positions). This suggests that the model’s distribution behaves slightly differently, likely due to more influence of conditional likelihood at the positions near the context. We will contain the figure in the camera-ready.

This observation might lead you to three questions.

[Q-1-1] If the model is simply mimicking the data distribution and therefore favoring high-prior or repeating tokens, why is this problematic?

= LLM is a probabilistic model that, when faced with high complexity, it often learns shortcuts to minimize loss. From the training data, the model learns that simply choosing easy tokens can always guarantee a least worth loss, even without contextual understanding.

We can formulate this problem as follows. A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). It can be exemplified as decoding $x_{21}, x_{22}, x_{23}$ at the first step, where $x_i$ denote a mask token that is i-th position from given context (similar to the “Lincoln” example). The probability of candidate $c$ for $x_{21}$ is $E_\tau [ p(\tau|c)p(c)]$ , where $\tau \in T$ is all possible combination of $\{x1,x2,...,x20\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), the infinity. Therefore, the posterior $p(c \mid \tau)$ converges to $p(c)$, where easy tokens dominate. This holds for $x_{22}, x_{23}$ as well,  which explains why patterns in candidate zone, such as “lincoln lincoln lincoln” emerge.

Such a problem is not a matter of “independently decoding” property. This is because, regardless of how much a model accounts for dependency with surrounding context on the output layer, no model can feasibly consider $|V|^{20}$ possibilities in advance.

[Q-1-2] On the other side, why do tokens like "Question" and "Answer", which were not even present in the training dataset, appear in the candidate zone?

= This is a good example of the stated problem in Q-1-1. The model chooses these candidates only because it is repetition, though it has no contextual meaning.

[Q-1-3] Then, what does R2FT do?

The motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is reducing $p(c)$ in $E[p(\tau|c)p(c)]$. By doing so, the posterior shifts toward tokens with higher average likelihood within T. In other words, we choose the candidate patterns like "president president president", which at least contains more contextual information. To avoid those three tokens decoded simultaneously (resulting in structural breakdown), possible strategies include 1) reducing T (e.g., semi-AR, Conv), 2) lowering the sampling rate, or 3) introducing revocability as you mentioned.

(We will show p(c) in each step for both SFT and R2FT in camera-ready, regarding your Question 3)

[Q-1-4] Isn’t it rather a problem of irrevocability and independence? Isn’t Conv and R2FT focused only on the Absorbing approach? (Absorbing vs uniform)

The issue you raised can also be framed as a debate between absorbing (i.e., MDLM, mask diffusion) and uniform approaches, as discussed in [1]. We anticipate that the absorbing family will remain prevalent, and revocability and dependency may be implemented in a way compatible with absorbing. That is why we concentrated on solving the problem of unrevocable MDLM first (i.e., “third way”). We provide the following rationales.

(1) First, the absorbing-based approach has already proved its downstream task performance comparable to AR LMs (e.g., LLADA). Regardless of its limitations, a method that already works this well is unlikely to be entirely abandoned. In contrast, uniform methods have been mainly evaluated on auxiliary metrics such as gen PPL.  We believe this stems from the following theoretical limitations.

(2) Revocation introduces an additional cost.

As the sample quality of diffusion LM is partially a function of sample rate $r=L/S$, revocability increases the number of masks to decode, similar to raising r. To investigate this, we compared the average unmasking per step (denote r*) of both revocable MDLM (reMDM [2]) and irrevocable ours.

ETable1. Average unmasking per step.

This shows r* of the revocable model is 2~3 times larger than that of MDLM, equivalent to operating with a step size 2–3 times smaller in MDLM, which causes degradation. This is one of the reasons that the downstream task performance of reMDM is also lower than MDLM (ETable2).

ETable2. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.

ETable3. GSM8k accuracy of LLADA-8B-Instruct based models. L=512, S=128, all CoT instructed, k=1.

(3) Revocability may not be a critical property.

As noted in [1], we often consider revocability as an inherent and necessary property of diffusion models (DM), thinking of the image generation service, transforming an existing image into an entirely different one (e.g., a person eating bread → a lion eating bread).

However, a closer look at the denoising process reveals that this is not as flexible as the entire output. To replace a person with a lion, the DM (1) first degrades the original image into an almost unrecognizable noisy state, and (2) then gradually denoise it. In the denoising phase, once strong edges or dots emerge at a certain location in the early steps, these elements tend to remain stable, functioning like anchors. Then, the surrounding blurred regions refine around them, determining how to interpret the anchors (e.g., as an eye or a nose). Thus, while the overall process is revocable, steps after initial destruction exhibit limited revocability. This may be related to the fact that the reverse process is more deterministic than explorative, since DDIM [3]. We will show visual examples in the camera-ready.

Likewise, existing successful generation models (e.g., AR) are not revocable. It may be that well-calibrated sampling can provide an enough solution.

(4) If revocability is still needed, a compatible way is promising.

[1] provide theoretical justification for this compatibility. In [1], the diffusion posterior $p(x | x_t)$ can be decomposed as $p(x|x_t) = p(N|x_t) p(x|x_t, N)$, where N is noise. The “uniform” approach models the entire, which is theoretically complete but practically beyond the model capacity. In contrast, the “absorbing” focuses only on modeling $p(x|x_t, N)$, which is theoretically incomplete but highly practical.

As an alternative, [1] proposes operating $p(N|x_t)$ independently while preserving absorbing as a main drive, achieving a balance between these two extremes. Remasking (ReMDM) [2] also lies along this stream. As long as absorbing models can operate independently, Conv and R2FT remain fully compatible, as also empirically shown in ETable2, 3.

---

[Q-2] R2FT’s rule-based approach is against the Bitter Lesson. (regarding Question 2)

(1) In R2FT, the rule-based component is only used for data construction, while the learning lies with the model itself. So R2FT is closer to SFT modeling than a rule-based system.

(2) R2FT can be viewed as a low-cost approach to human data annotation.

Creating datasets for desired model behavior has been a guaranteed approach in LLM development. Research fields are further mining new datasets to broaden model coverage. Likewise, we can annotate “undesired” data to prevent unwanted behaviors. However, as “deconstruction is always much simpler than construction**”**, unwanted behaviors can often be easily reproduced through simple rules, which is R2FT.

---

[Q-3] When uni/bi directional Conv is used?

In this work, we employed unidirectional sampling exclusively, as bidirectional sampling did not yield significant performance differences. This is likely because the context is presented only on the left side in QA tasks. We noted this and discussed the possible advantage of policy optimization in the limitation section.

---

[1] Liu et al. Think while You Generate: Discrete Diffusion with Planned Denoising

[2] Guanghan Wang et al., Remasking Discrete Diffusion Models with Inference-Time Scaling

It's a pleasure to reconnect with you! Thank you for the thoughtful discussion. We marked “REQUESTED ACTION” on the according questions. We are happy to engage with any questions or discussions until the end of the discussion period.

---

[Q] The issue arises when sampling multiple tokens at once, not when sampling a single token.

= We agree. Diffusion models inherently aim to sample multiple tokens to gain speed, which introduces this problem. As noted at the end of Q-1-3, we suggested several ways to mitigate it: (1) increase S (i.e., step size) (2) sampling from only closer positions (e.g., Conv, semi-AR) (3) or introduce revocability.

As you noted, introducing interdependent decoding would also be an effective solution.

---

[Q, REQUESTED ACTION] The meaning of ''Such a problem is not a matter of “independently decoding” property.''

= It seems the difference in perspective comes from the examples we each assumed; our focus was on the early steps.

Case1. Example assumed in the Copula paper [1].

The <MASK> dog <MASK> the neighbors.

The example assumed in Copula (Case1, from Figure 1 in [1]) involves mid-timesteps, where a substantial amount of masks are already decoded and the remaining possibilities are significantly constrained. In such cases, we agree that a Copula could serve as an effective solution.

In contrast, the example we assumed (Q-1-1) is sampling at early steps over an empty space. In this case, the number of variables to consider is extremely high, making it ineffective to precompute dependencies.

To state it more formally, consider a model designed to capture dependencies by computing the conditional joint probability $p(x_{21}, x_{22} | \tau)$, rather than only $p(x_{21} | \tau)$ as in the Q-1-1 example. However, if we assume $\tau$ is in $T$, all possible combination of candidates of masked tokens $\{x_1 \dots x_{20}\}$, then $E_\tau [p(x_{21}, x_{22} | \tau)]$ is marginalized into $p(x_{21}, x_{22})$, from the same mechanism in Q-1-1, leaving no contextual information available. This is what we ment by ''Such a problem is not a matter of “independently decoding” property''. This necessitates additional solutions to address the concern (e.g., R2FT, or revocability).

---

[Q] Could the “Copula” be a promising direction?

= We agree that if a method effectively captures dependencies, it would be an excellent advance. We also find the Copula [1] highly interesting and a promising direction. Furthermore, its design is compatible with the absorbing approach, making it also compatible with our method. However, we have the following concerns:

Before explaining our concerns, let us first clarify our understanding of the Copula model. While we often simply state that diffusion LMs predict $p(x_t |x_{t+1})$ independently of other positions, this is only partially true. In practice, it already incorporates aspects of cross-correlation due to the transformer’s cross-attention mechanism. Therefore, computing cross-correlations of model outputs (e.g., Copula) can essentially be interpreted as feeding the outputs into an additional transformer-like layer. Indeed, the implementation of Copula feeds the model outputs into a separate transformer.

Viewed this way, the following concerns arise:

(1) The conditional probability $p(c∣\tau)$ ($c$ is candidates) computed by the model indicates the model’s knowledge itself: whether puppy is associated with scared or calmed depends on the distribution of its training data and experience. However, if a separate transformer determines $p(c∣\tau)$, the final output may become misaligned with the knowledge of the base model.

In this reason, if the Copula model is smaller in size, it may force to generate outputs with less knowledge than the base model. While Copula has demonstrated strong performance in evaluation of structural coherence, it could be not true in downstream tasks requiring extensive knowledge.

(2) The most direct way to address the issue (1) is to scale up the Copula model so that it can absorb sufficient knowledge. However, as the Copula model grows larger, it has little difference from simply doubling the step size.

This appears to be a critical concern. In fact, reviewing the Copula paper and its code scripts, we found that a single sampling step requires three separate model inferences (twice for 320M, and once for 137M). Thus, one step of Copula should be compared with 2.5 steps of original model (e.g. S=128 vs S=256–512). However, as you mentioned, increasing the step size alone mitigates much of the dependency issue.

Nevertheless, we believe this approach is highly intriguing and potentially helpful after the mid-step.

Thank you for the insightful question. We will incorporate these discussions into the camera-ready version.

---

[Weakness 1] R2FT relies on simple rule-based negative sample generation; more sophisticated negative mining methods can be considered.

= The point you raised remains an avenue for future research. However, it can be justified as follows.

(1) We already discussed the strengths of our approach compared to other seemingly more natural alternatives in section 4.2

(2) R2FT can be viewed as a low-cost approach to human data annotation.

Creating datasets for desired model behavior has been a guaranteed approach in LLM development.  Furthermore, the ongoing progress in LLMs relies heavily on continuously generating new datasets to broaden the model coverage. Likewise, we annotate new data for LLMs to prevent unwanted behaviors. However, as “deconstruction is always much simpler than construction**”**, unwanted behaviors can often be easily reproduced through simple rules, which is R2FT.

---

[Weakness 4] It's not clear how exactly the performance improvement comes from, does it primarily stem from scenarios requiring long contextual information?

= If the intent of this question is to inquire about the theoretical guarantees of our method, we would kindly ask you to refer to our response to QCLw for further details. We apologize for not including the full details here, as we did not want to occupy excessive space if this was not your intended concern.

---

[Weakness 3,4 ] The results are purely based on win rate, lacking analyses of more diverse dimensions. \ It's not clear how exactly the performance improvement comes from; does it primarily stem from scenarios requiring long contextual information?

= Acknowledging your concern, we provide an additional metric. Text quality involves two key aspects: alignment and structural coherence. Among these, alignment is primarily assessed using win rate. Besides, s structural coherence is typically assessed using generative perplexity (gen PPL). However, gen PPL is known to overestimate texts containing repeated patterns [1], which makes it easy to be gamed.

Accordingly, we computed the mean ($\mu$) and stds ($\sigma$) of gen PPL from the training dataset, and then measured the proportion of model-generated texts whose PPL falls within the interval $\mu \pm 2\sigma$ (reported as inlier in ETable 1). Additionally, we report the count of outputs which has zero length before the EOS token, indicating another pattern of structural incoherence.

Furthermore, as you mentioned, our method assumes scenarios where long-form text must be generated, without relying on random unaligned sampling, following recent expectations for LLM-based services. Thus, we conducted additional experiments under a setting designed to better reflect this assumption. To reflect this setting, we instructed models to generate responses in a Chain-of-Thought format to provide detailed reasoning, and we applied top-5 decoding. As average length increased, we set S=128, which still guarantees around 3x sample rate.

ETable 1. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.

Results in new setting are consistent with previous one. Furthermore, this multi-dimensional evaluation reveals stronger evidence for our hypothesis about the proposed method, which is explained in the next question.

---

[Question 1] I suggest the authors conduct a more fine-grained evaluation, analyzing the quality and contextual relevance of generated tokens as a function of their distance from the prompt/context.

= Contextual relevance as a function of distance from the prompt is basically illustrated in Figure 3. The probability of candidate $c$ on the given context $x$ is $p(c|x)=p(x|c) p(c)$, where $p(c)$ represents a frequency without any contextual information. This term corresponds to the high-prior tokens or naive repetition, that we identified as problematic (Section 2.2). Figure 3 visualizes $p(c)$, showing that p(c) peaks at the 5–10 steps away from the context, then remains at a high level thereafter (30–50%). This suggests that the influence of the contextual term $p(x∣c)$ relatively diminishes substantially with increased distance.

R2FT explicitly reduces $p(c)$, which leads to the relative increase of $p(x|c)$, which leads to contextually more aligned sampling. As shown in ETable 1, our multi-dimensional evaluation of alignment (win rate) and structural coherence (inlier) captures these trends well. First, categorical sampling with SFT alone (with high $p(c)$) achieves enough structural coherence (inlier=1.00) but low alignment. More deterministic Topk-k decoding has better alignment, but risks structural degradation. In contrast, R2FT results in a noticeable improvement in both alignment and structural coherence.

---

[Weakness 2] In most cases, window size is set to1024(512 in some experiments) , it's hard to judge the robustness/generalization of the method to the window size.

= Thank you for pointing out this important aspect. We are preparing the experiment and will provide the results during the discussion period if possible.

---

[Question 2] One more thing, I was also curious about the performance on bi-directional substream tasks, as mentioned by the authors in the limitations, it would be good if they can provide some basic results.

= As you point out, we did not evaluate on bi-directional tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.

However, we believe that the bidirectionality introduced by Conv offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.

In our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.

We also expect Decision Transformers (DT) [2] to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.

However, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints. In this scenario, Conv can be a better option than a more rigid semi-AR.

---

Reference

[1] A. Holtzman et al., The curious case of neural text degeneration

[2] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling

We sincerely appreciate your positive assessment of the strengths of our work. We will incorporate all of the discussions below into the camera-ready version, along with your suggestions on citations and writing.

---

[Weakness 1] Claim that semi-AR sacrifices speed may not hold when KV-cache is used in diffusion models.

= We agree with this point. However, the speed gains from KV-cache in semi-AR models are compensated by the increased cost from timespan expansion, implying that diffusion LMs cannot fully leverage their inherent speed advantage with semi-AR.

In contrast, Conv can also theoretically leverage KV-cache, as the boundary of the mask tokens shifts consistently, as illustrated in Figure 7. For this reason, Conv can be considered to have a speed advantage.

---

[Question 1] Line 135 states that sampling from top-ranked candidates “high chance of selecting only repetition or high-prior tokens positions far from context.” Why? Do high-probability tokens actually cluster near the prefix, as suggested by darker colors in Figure 2?

= A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). To formalize this, consider a scenario in which the model decodes $x_{21}, x_{22}, x_{23}$ at the first sampling step, where $x_i$ denote a mask token that is i-th position from given context. The probability of candidate $c$ for $x_{21}$ is $p(c|\tau)=E_\tau [ p(\tau|c)p(c)]$, where $\tau \in T$ is all possible combination of $\{x1,x2,...,x20\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), the infinity. Therefore, the posterior $p(c \mid \tau)$ converges to $p(c)$. This is consistent with the empirical observation in Figure 3, where the probability of high-prior or repetitive tokens increases and then plateaus at a high level as the distance from the given context grows.

$p(c)$ represents the probability of a candidate that can be easily predicted without any contextual information. Typical examples include high-prior tokens (e.g., function words) or simple repetition of the given context. The same applies to $x_{22}$ and $x_{23}$. Consequently, as illustrated in the upper part of Figure 11, top candidates often include patterns such as “: : :” (repetition) or “the the the” (high-prior tokens).

The motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is training the model to reduce $p(c)$ in $E[p(\tau|c)p(c)]$. By doing so, the probability mass shifts toward tokens with higher average likelihood within $T$. In other words, we choose the candidate patterns like "president president president", which at least contains more contextual information than ": : :". This is consistent with our empirical observation on AlpacaEval that R2FT achieves better alignment with the query.

---

[Weakness 2] All analyses in Section3 use relatively small MDLMs. It’s unclear if results hold for larger models (e.g., 7B LLaDA).

= That is a valid point. We are preparing the experiment and will share the results during the discussion period.

---

[Weakness 3] Convolution direction needs to be clarified. Uni-directional convolution mimics causal decoding in AR models. Which convolution type is used in the end, uni- or bi-directional?

= The bidirectional decoding we referred to applies to scenarios where context is provided on both sides (left and right), unlike simple QA settings, and we did not evaluate on such tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.

However, we believe that the bidirectionality introduced by Conv offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.

In our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.

We also expect Decision Transformers (DT) [2] to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.

However, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints. In this scenario, Conv can be a better option than a more rigid semi-AR.

---

References

[1] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling NeurIPS 2021

Thank you for the careful review. We would like to offer the following clarification.

---

(1) Diffusion LM has bidirectional attention.

To provide a clearer explanation, we distinguish between “bidirectional attention” and “bidirectional generation”.

AR models inherently employ unidirectional attention in their architecture, whereas diffusion-based LM architectures leverage bidirectional attention, which applies to both Conv and semi-AR.

Therefore, even when diffusion LMs perform unidirectional generation, they continue to maintain locally bidirectional attention. This applies to both semi-AR and Conv variants.

For example, consider the prompt:

“Who is the president: __________”

If, in the next step, a token happens to be sampled in the middle, resulting in:

“Who is the president: ______is __”,

then the masked positions between “president:” and “is” will attend to context on both sides (e.g., “president:”, “is”). Thus, bidirectional attention occurs in a local level.

This behavior can be observed in Figure 7(a), where sampling does not proceed strictly sequentially as in AR but occurs in a scattered manner. The mask tokens between scattered tokens are applied with bidirectional attention. This bidirectionality is a difference from AR models.

However, this situation (activating bidirectional attention) is largely a byproduct of the model sampling multiple positions in a scattered manner, which might provide little benefit in most of the existing LLM tasks. This is because, in most LLM tasks (e.g., QA), useful information is mainly provided on the left side of the window. In fact, this property can even introduce the LDW problem, where tokens sampled far from the given context become misaligned, forcing harmful bidirectional attention.

$\color{white}$

(2) Conv is capable of bidirectional generation.

“Bidirectional generation” refers to the overall direction in which generation is globally grounded. Semi-AR enforces a strict rule to proceed from left to right, whereas Conv, when context is provided on both sides, automatically grounds generation multi-directionally. This distinction is illustrated in Figure 7.

Tasks that provide context on both sides are goal-oriented, as explained. For example, negotiation requires aligning with previous utterances and desired responses. And even in QA tasks, desired responses or scores can be framed as explicit goals. This naturally offers a way to reinforce LLMs toward alignment with annotated preferences. This is also the scenario where the bidirectional attention of diffusion LMs provides a clear benefit.

However, as noted in our limitations, such bi-context tasks are currently rare—likely because most LLM tasks have historically been designed for AR-based models. If diffusion LMs gain broader adoption, we anticipate that more of these tasks will emerge.

Therefore, we believe that solving such tasks represents one of the key motivations behind the growing interest in diffusion LMs, which is why we chose to report the bidirectional generation property of Conv.

Of course, this claim has not been empirically validated due to the absence of such tasks. We have noted this as a limitation and believe that explaining the underlying principles of Conv sufficiently conveyed our setting without misleading. To avoid any misleading, we will emphasize this point more clearly in the revised version.

$\color{white}$

#. Why is Conv valuable, and how does it differ from AR?

Without bidirectional generation, Conv still offers a superior approach compared to the previous SOTA method (semi-AR) for addressing the LDW problem, which is our main claim for contribution.

As you noted, Conv enforces a generation process grounded in the provided context, similar to AR. However, this mechanism serves a specific purpose: mitigating the LDW issue.

Default diffusion LMs often generate tokens at distant positions, leading to outputs that are poorly aligned with the given context (Section 2). In contrast, AR avoids LDW because it only unmasks the position attached to the context. Semi-AR, the previous SOTA method, mimics this benefit of AR by strictly partitioning the decoding space into blocks and processing them left-to-right (i.e., autoregressively). Yet, this approach introduces a time-interval expansion problem, negating the speed advantage over AR.

Conv, by comparison, narrows the context to reduce LDW like semi-AR, while avoiding the time-interval expansion issue, as we provided both theoretical guarantees (Response to reviewer QCLw) and empirical evidences (Figure8, 9). As a result, Conv mitigates LDW without sacrificing the decoding speed advantage, resulting in faster speed than AR—a key distinction from both AR and semi-AR.

We greatly appreciate the reviewer’s acknowledgment of our work’s strengths and the insightful questions that contribute to its robustness. These discussions will be incorporated into the camera-ready submission.

---

[Question2-1] Theoretical guarantee for why Conv avoids the time-interval expansion problem.

= We provide two theoretical explanations for why Conv may outperform semi-AR. The following discussion assumes a mask diffusion type [1].

(1) Violation of the training assumption.

This point is discussed in detail in Section 3.2 (line 174).

(2) Explanation with the hazard function.

A more intuitive explanation is that Conv is more flexible than semi-AR, granting the model greater freedom. This can be theoretically justified using the hazard function, which measures the probability of corruption.

Let us follow the notation in the paper, line 174: L as window size, S as step size, b as the number of blocks, $L_b$ as the block size or kernel size, and $S_b$ as the number of steps assigned for each block in semi-AR.

We newly define the sample rate $r=L/S$, and $W$ as the number of mask tokens that the model have to sample from at current step. For example for default diffusion LM, $W=L$ holds at the first decoding step, and $W \approx L-r$ at the second step, since the model might have unmasked $r$ masks at previous step, on average. For semi-AR, it is $L_b$ to $L_b - r$.

We further define the hazard function $P$, representing the probability (or degree) that the final decoded text becomes structurally (grammatically, syntactically) harmed. Conversely, $1-P$ representes the probability of being unharmed (i.e., structurally coherent). We define $Q = \log(1-P)$. And let small $p_t$ denote the probability that the final output becomes harmed due to the decoding at intermediate timestep $t$. Here, the hazard function is commonly defined as $P= 1- \prod_{t=1}^S (1-p_t)$, which in turn $Q= \sum_{t=1}^S \log(1-p_t)$. We assume that $p_t$ depends on both $r$ and $W$, thus we denote $p_t=p_r(W)$.

#. Assumptions about the factor in the quality of a text.

We assume that the quality of generated text is related to density of decoding (denoted as ****$d=r/W$) in two opposing ways (though there can be more other factors). Raising $d$ can be benefit and harm the quality at the same time.

(1) Alignment:  When the positions to be decoded are far from the given context, the model tends to suggest less aligned candidates—a phenomenon referred to as the LDW problem**.** This situation is often associated with low $d$ (though not always), which indicates lots of masked positions are between the given context and the targeted position (refer to response of Question2-2). A common strategy to mitigate this issue is to increase $d$ by reducing $W$ (e.g., semi-AR, Conv).

(2) Structural coherence: Conversely, when $d = \frac{r}{W}$ becomes too high, it can reduce the dependency among decoded tokens and lead to structural degradation. This effect stems from a key property of mask diffusion: multiple tokens generated simultaneously cannot attend to one another at the output layer, resulting in low inter-token dependency. Consequently, when many tokens are generated within a narrow span, the likelihood of syntactic conflicts increases.

However, when the distance between two generated tokens $x_i$ and $x_j$ is large “enough”, even if their immediate dependency is weak, the number of possible connecting sequences between them grows exponentially as $|V|^{(j-i)}$ (V: vocab size). This implies that later decoding steps have the opportunity to restore structural connectivity. This is why $p_t$ depends on $r$ and $W$. If large r and small W mean too packed generation, thus low $p_t$.

We formalize this property as follows:

$\text{if } r_1 = r_2 \text{ and } W_1 > W_2, \text{ then } p_{r_1}(W_1) \leq p_{r_2}(W_2)$

Equality approximately holds when $r$ is sufficiently small or when both $W_1, W_2$ are sufficiently large over some threshold. The precise threshold for “sufficiently” and the shape of the curve $p_r(W)$ as a function of $W$ may vary across models.

#. Comparison Across Decoding Types

From the two factors discussed earlier, we now focus on (2) structural robustness and model it using the hazard function across different decoding strategies, leaving (1) aside for simplicity.

1. Default mask modeling

For a standard mask diffusion model without a narrowed decoding window, the hazard function $Q_0$ is as follows.

$P \approx 1-\Pi_{t=1}^{S}(1-p_r(L-(t-1) \cdot r))=1-\Pi_t^{S}(1-p_r(t \cdot r))$

$Q_0 = \sum_t^S q_r(t \cdot r)$ , $q = log(1-p)$ , $W_t = t\cdot r \leq L$

$q_r(W)$ represents the log probability that decoding at a step does not introduce corruption. $W$ decreases gradually by $r$ per step.

1. semi-AR

For semi-AR, when $L$ is divided into 4 blocks (i.e., $L_b = 128$):

$Q_{sa}=b*\sum_t^{S_b}q_r( t \cdot r)$ , $W_t = t \cdot r \leq L_b$

To compare, since $S > S_b$, the value of $q_r(W_t)$ for semi-AR at any given step is strictly smaller than that of the default setting (because $W_t \leq L_b \leq L$ for semi-AR).

Therefore: $Q_{sa} \leq Q_0$

Equality occurs when the block size $L_b$ approaches $L$, or when $S$ is large enough so that the argument of $p_r(W)$ exceeds its threshold for most steps. This theoretical behavior aligns with the empirical observation in Figure 4, where evaluated structural quality increase as $L_b$ or $S$ are raised.

1. Conv

For Conv, assume the kernel size $L_b=128$. In this case, at every step we have $W\geq 128$. Although the actual variation of $W$ is more complex, for simplicity, we assume $W = 128$.

$Q_c = (L-128) * q_r(128) + \sum_t^{128/r}q(t \cdot r)$

Here, the second term is shared with semi-AR and the default. but the first term is always larger than that of semi-AR, except at the start point of each block.

Therefore, $Q_{sa} \geq Q_b$

As before, equality approximately holds when $S$ is sufficiently large.

Compared to the default mask diffusion, the first term is always smaller than the default, so we have: $Q_c \leq Q_0$.

Equality holds when the fixed kernel size is “sufficiently” large, so that the effective $W$ go over threshold. This behavior is consistent with the empirical observation shown in Figure8, where kernel size over some threshold yields structural stableness.

To wrap up, In terms of structural robustness, we generally observe the ordering:

$\text{semi-AR} \prec \text{conv}\prec \text{default}.$

However, from the perspective of alignment, the relationship reverses:

$\text{semi-AR}, \text{conv} \succ \text{default}.$

Consequently, adopting Conv offers a favorable trade-off, as it provides substantial gains in alignment while maintaining significantly better structural robustness compared to semi-AR.

This argument is consistent with our empirical findings. Categorical sampling has good structural coherence (inlier in ETable1), while alignment (winrate) degrades. Conv has better alignment and winrate, while semi-AR fails in both.

ETable 1. AlpacaEval of LLADA-8B-Base based models. L=512, S=128, all CoT instructed, k=5 in topk.

---

[Question2-2] Theoretical guarantee for why R2FT effectively mitigates.

=We divide the explanation into two steps: the decoding step and the training step.

(1) Decoding step

First, let us describe the scenario where the issue arises. A key characteristic of diffusion LMs is decoding tokens that are far from the immediate context (LDW problem). To formalize this, consider a scenario in which the model decodes $x_{21}, x_{22}, x_{23}$ at the first sampling step, where $x_i$ denote a mask token that is i-th position from given context. The probability of candidate $c$ for $x_{21}$ is $p(c|\tau)=E_\tau [ p(\tau|c)p(c)]$, where $\tau \in T$ is all possible combinations of $\{x1,x2,...,x20\}$. The cardinality of T is $|V|^{20}$ (V is vocab size), infinity. Therefore, the posterior $p(c \mid \tau)$ converges to $p(c)$.

p(c) represents the probability of a candidate that can be easily predicted without any contextual information. Typical examples include high-prior tokens (e.g., function words) or simple repetition of given context. The same applies to $x_{22}$ and $x_{23}$. Consequently, as illustrated in the upper part of Figure 11, top candidates often include patterns such as “: : :” (repetition) or “the the the” (high-prior tokens).

The motivation of R2FT is: if guessing is inevitable, let it be at least more aligned. Formally, this is training the model to reduce $p(c)$ in $E[p(\tau|c)p(c)]$. By doing so, the probability mass shifts toward tokens with higher average likelihood within $T$. In other words, we choose the candidate patterns like "president president president", which at least contains more contextual information than ": : :". This is consistent with our empirical observation on AlpacaEval that R2FT achieves better alignment with the query.

(2) Training step

Rationale on reducing p(c) is described in Section 4.2

---

[Question 1] Failure cases

(1) Conv: As discussed above, Conv exhibits reduced structural coherence when the kernel size or the step size becomes excessively small.

(2) R2FT: Without incorporating the NELBO term during training, R2FT tends to eliminate high-prior tokens entirely. However, this issue is resolved by adding the NELBO term and setting $\gamma = 0.1$.

Such cases will be included in the camera-ready version.

---

[Question 3] GPU usage and time

We will include in the camera-ready version or report in the discussion period.

Thank you for raising such valuable questions that make our work more robust. We will incorporate all related discussions into the camera-ready version.

---

[Q1] The paper is dense and somewhat hard to follow in parts. Lots can be done here to make it easier to follow and less dense. Paper is too packed. Figures are too small. This limits readability and potential impact of this work.

= We appreciate your observation. The formatting likely became dense as we attempted to cover all the necessary details. We will improve readability in the camera-ready version.

---

[Q2] Some response samples still show factual or stylistic oddities.

= That is correct. While our method improves performance over previous approaches, diffusion LMs still face several inherent limitations—most notably, sensitivity to the step size ($S$) and the lack of revocability. The latter issue has been partially addressed by recent work such as reMDM [1], which introduces remasking strategies, that are also compatible with our approach. Exploring such limitation remains an important direction for future research.

---

[Q3] Evaluation on tasks like reasoning or structured output is limited. Can Conv methid generalize to structured generation tasks?

How sensitive are the methods to kernel size?

= Conv generalize to structured generation tasks, significantly outperforming the competitive baseline LLADA on reasoning tasks (GSM8K, MMLU) (ETable 1-2). And Conv also exhibits minimal sensitivity to kernel size even in small step size ($S = 64$).

ETable 1. GSM8k with LLADA-8B-instruct , L=512, S=64, CoT instruct. Each column is kernel size or block size

ETable 2. MMLU with LLADA-8B-instruct , L=512, S=64, CoT instruct. Each column is kernel size or block size

---

[Q4] How sensitive is the R2FT to the corruption rule?

= As shown in Algorithm 1, the corruption rule conducts random sampling based on a few predefined rules. Consequently, slight variations of hyperparameters in these rules appeared to have minimal impact on performance, and we were able to reach strong results without extensive tuning of these rules.

However, the training objective shows more sensitivity to the hyperparameters $\gamma$ and $\beta$. Excessive unlearning of unpreferred cases can lead to the elimination of high prior tokens, thereby removing essential functioning words (e.g., is, a). This issue appears to stem from the nature of the unlearning process itself. Nevertheless, we addressed it by incorporating an additional objective term $\mathcal{L}_{NELBO}$ and setting $\gamma = 0.1$, which mitigated this problem effectively.

---

[Q5] Weak investigation of bidirectional downstream tasks, despite bidirectionality being a central claim.

=As you point out, we did not evaluate on bi-directional tasks in our paper. We noted this in the Limitations section and deliberately avoided strongly emphasizing bidirectionality as a major advantage. If you feel that it was overstated, we will further tone down the discussion in the revision.

However, we believe that the bidirectionality introduced by Conv offers a highly significant advantage, and we hope future researchers will draw inspiration from this aspect. That is why we made a deliberate effort to mention it, even if only briefly.

In our view, the most impactful application of bidirectionality of Conv lies in tasks that need goal awareness, which requires generation grounding on both previous context and preferred reaction (e.g., negotiation). However, designing such tasks is beyond the current scope, so we leave it for future work.

We also expect Decision Transformers (DT) [2] to be highly synergetic with Conv, which is a policy model with LLM architecture. Current DT predominantly rely on AR models, which are inherently unidirectional. This limitation makes them goal-unaware. As a result, during exploration for RL, they often perform numerous random, goal-agnostic actions, significantly reducing the efficiency of trajectory sampling. In this context, DT with diffusion LM architecture can provide a major breakthrough by enabling more goal-oriented sampling.

However, even in this setting (especially in the long-horizon scenario), bidirectional DTs may still suffer from the LDW problem, and decoding that is grounded in the adjacent context from both sides (start point and goal point) would be particularly advantageous in this setting. It becomes nearly impossible to effectively ground when the trajectory involves multiple intermediate waypoints. In this scenario, Conv can be a better option than a more rigid semi-AR.

---

References

[1] Guanghan Wang et al., Remasking Discrete Diffusion Models with Inference-Time Scaling, 2025 arxiv

[2] Lil Chen et al., Decision Transformer: Reinforcement Learning via Sequence Modeling NeurIPS 2021