Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation

We thank the reviewer bsFC for the insightful feedback and clarify their concerns below.

Discussion on formal approximation bound for the Top-H

While the primary objective of our work was to empirically demonstrate the efficacy of the top-H decoding, we largely intend to keep the detailed theoretical underpinning of the error bound analysis as a part of future research. Nevertheless, we now attempt to demonstrate a worst-case bound for top-H under the specific assumption of a probability distribution.

Zipf model. Fix a vocabulary of size $n$ and exponent $s>1$. We assume the sorted next-token probabilities obey the classical Zipf / regularly varying law:

$p_i := \frac{i^{-s}}{H_{n,s}},\qquad H_{n,s}=\sum_{j=1}^{n}j^{-s}. \quad (1)$

Empirically, language-model logits are well approximated by $s\in[1.05,1.20]$ [1]; hence the assumption captures current practice.

Why a Zipf assumption matters. Because ECMM is NP‑hard (Appendix C), exact polynomial‑time solutions are unlikely. In the absence of structural assumptions, a constant-factor approximation guarantee for ECMM is highly unlikely unless P=NP, as our NP-hardness proof relies on a gap-preserving reduction from Cardinality-Constrained Subset Sum. This structure is known to be fundamentally difficult to approximate, a standard result in computational complexity theory [2]. However, LLM logits consistently follow heavy‑tailed (Zipf / regularly‑varying) laws, so analysing this regime yields practically relevant bounds.

Notation. We write $M(k)=\sum_{i\le k}p_i$ for the prefix mass, $T(k)=1-M(k)$ for the tail mass, and $H(k)$ for the entropy of the normalized prefix distribution $q^{(k)}_i=p_i/M(k)$.

---

Preliminaries

Lemma 1 (Monotonicity of prefix entropy). For $1 \leq k < n$, $H(k) < H(k+1)$.

Proof. Please refer to the proof for Theorem 3 of the main manuscript.

Lemma 2 (Tail mass bound). For $k<n$,

$\frac{(k+1)^{1-s}-n^{1-s}}{(s-1)H_{n,s}} \le T_{n}(k)\le\frac{k^{1-s}}{(s-1)H_{n,s}}.$

When $n\gg k$, the numerator difference is $o(k^{1-s})$, so the upper bound is asymptotically tight.

Proof. Apply upper and lower Riemann sums to $\int_{k}^{n}x^{-s}\,dx$.

Lemma 3 (Prefix entropy asymptotics). There exist constants $c_s,C_s>0$ such that

$c_s+\frac{s}{s-1}\log\left(\frac{k}{2}\right)\le H(k)\le C_s+\frac{s}{s-1}\log k \quad \text{for all } k\ge 2.$

Proof. Combine Lemma 2 with integral bounds for $\sum i^{-s}\log i$.

---

Depth of the greedy prefix

Let $k_g:=\max\{k:H(k)\le \alpha H(n)\}$. Lemma 1 implies $k_g$ is well defined.

Lemma 4 (Growth rate of $k_g$). There exist constants $a_s,b_s>0$ such that $a_s n^{\alpha}\le k_g\le b_s n^{\alpha}$.

Proof. Insert Lemma 3 into the defining inequality and solve for $k_g$.

---

Mass captured by Top-H

Write $\Gamma_g=M(k_g)$. This is the mass captured by the greedy solution. By construction, this solution is valid as it satisfies the entropy constraint $H(k_g) \le \alpha H(n)$.

Tight upper bound for ECMM

Let $S^\star$ be any subset satisfying the entropy constraint, and set $\Gamma_\star=\sum_{i\in S^\star}p_i$. We want to bound the maximum possible value of $\Gamma_\star$.

The greedy algorithm selects the prefix $[k_g]$ and captures a mass of $\Gamma_g = M(k_g) = 1 - T(k_g)$. The total mass of all tokens not in the greedy solution is, by definition, the tail mass $T(k_g)$.

Any other valid solution $S^\star$ can, at best, capture the mass of the greedy solution plus some portion of the remaining tail mass. The absolute maximum mass any solution can capture is 1 (the entire vocabulary). Therefore, the maximum possible improvement any optimal solution $\Gamma_\star$ can have over the greedy solution $\Gamma_g$ is bounded by the tail mass that the greedy algorithm left behind:

$\Gamma_\star - \Gamma_g \le 1 - \Gamma_g = 1 - M(k_g) = T(k_g). \quad (\dagger)$

This gives us a direct upper bound on the additive gap between the optimal and greedy solutions.

---

Theorem 1 (Distribution-dependent additive guarantee). Under equation (1), for every $n\ge 4$,

$\Gamma_\star - \Gamma_g \le T_{n}(k_g) \le \frac{k_g^{1-s}}{(s-1)H_{n,s}} = \mathcal{O}(n^{-\alpha(s-1)}).$

Proof. From ($\dagger$) we have the additive gap $\Gamma_\star-\Gamma_g\le T(k_g)=T_{n}(k_g)$. Lemma 2 bounds $T_{n}(k_g)$ by $\frac{k_g^{1-s}}{(s-1)H_{n,s}}$. Finally, Lemma 4 gives $k_g=\Theta(n^{\alpha})$, so the gap decays as $\mathcal{O}(n^{-\alpha(s-1)})$.

Discussion on the effectiveness of greedy:

We understand that the approach of dropping high-probability tokens could in principle beat the greedy prefix by allowing more low-mass tokens from the tail. However, as discussed in the paper, this problem (ECMM) is NP hard, and we approximate the solution via a practically feasible greedy approach. Notably, we empirically demonstrate the effectiveness of this greedy based top-H approach to be superior to the existing SoTA. Additionally, under the constrained distribution of the Zipfian regime, the greedy prefix is constructed to generally maximize mass under minimal entropy growth. Because entropy increases monotonically with each added token, and the most probable tokens usually contribute less to entropy per unit mass, the greedy approach reaches the entropy threshold more efficiently than any alternative.

Comparison of top-H to other entropy-based sampling method:

The manuscript has results with $\eta$-sampling. We now present more comparison with Mirostat [3] on MTBench (R3T1), and GPQA (R3T2) as an additional entropy‑aware baseline. Unless otherwise noted, all decoding/eval settings match the main paper; we set $\tau$ = 3 for Mirostat.

R3T1

Top‑H wins in 9/9 settings. Averaged over all models and temperatures, Top‑H = 6.163 vs. Mirostat = 5.439 (+0.724 absolute, +13.3%).

R3T2

GPQA: Top‑H wins 7/9 settings (1 Mirostat win at LLaMA‑8B, T=1.0). Averages: Top‑H = 29.71 vs. Mirostat = 28.48 (+1.23 absolute, +4.3%).

Computational cost of top-H Please refer to the table R1T1 and corresponding response to reviewer ELnF .

Human Evaluation results Please refer to the table R1T2 and corresponding response to reviewer ELnF .

---

References

[1] Gerlach, Martin, and Eduardo G. Altmann. "Stochastic model for the vocabulary growth in natural languages." Physical Review X 3.2 (2013): 021006.

[2] Papadimitriou, Christos H. "Computational complexity." Encyclopedia of computer science. 2003. 260-265.

[3] Mirostat: A neural text decoding algorithm that directly controls perplexity, arXiv.

Dear Reviewer,

Thank you for the time and effort you’ve put into reviewing our submission. We’ve carefully considered each of your comments and concerns, and have done our best to address them thoroughly in our rebuttal with the necessary theoretical or empirical evidences.

We understand that time-bound reviewing process is demanding. However, as the deadline of discussion is approaching soon, we would greatly appreciate it if you could take a moment to revisit our response and, if possible, review the scores. With the enhanced thoroughness driven by your feedback, we believe top-H decoding has emerged as a promising alternative to the existing approaches.

Your feedback is invaluable and can help ensure that our clarifications are appropriately considered in the final decision.

Please don’t hesitate to reach us out in case any further discussion is needed.

Best regards,

Authors

We thank the reviewer 2JSm for the insightful feedback and clarify their concerns below.

Inadequate statistical reporting:

We have performed 10 runs and computed standard deviation across all runs for each config. We see the max std dev to be $\pm 0.35$; we commit to update Table 3 with the detailed results.

Validation with large model:

We replicated the MT-Bench (R2T1) and GPQA (R2T2) experiments using LLaMA3.3-70B-Instruct with the setup of section 6.1 in the paper. Specifically, top-H outperforms min-p by up to 6.0% and 6.47% on MT-Bench and GPQA, respectively.

R2T1

R2T2

Additionally, with the large model, we produced results with LLM-as-judge setup described in Section 6.1.3, and is reported in Table R2T3. This demonstrates top-H’s consistent improvement trend over alternatives.

R2T3

Human evaluation results:

Please refer to the table R1T2 of the rebuttal response for reviewer ELnF.

On misguided problem formulation-explanation of greedy solution to not be optimal:

While the reviewer’s intuition largely holds in practice, there are certain situations where the greedy solution wouldn’t be optimal and there are some nuances only ECMM captures. We explain it below.

Case for a lower-p token might be preferred over a higher one: Entropy responds in a highly non-linear way after renormalization. When the entropy cap is tight and we have multiple high-probability (dominant) tokens, the distribution can be so flat that adding any extra token violates $H(q)\\le \\alpha H(p)$. Dropping one of the dominant tokens sharpens $q$, letting us include several medium-probability tokens whose combined mass exceeds what was lost, thereby increasing $\\Gamma\_S$. We present an example of such in sorted order, resembling an hypothetical set of probabilities generated by an LLM.

[4.438e-01, 4.006e-01, 1.027e-01, 3.320e-02,
 8.733e-03, 5.801e-03, 2.413e-03, 1.468e-03,
 4.460e-04, 4.286e-04, 1.804e-04, 5.383e-06,
 5.205e-07, 3.760e-08, 1.521e-12]

Let the entropy threshold with $\alpha=0.4$ be $0.6796$. The greedy solution takes only the first token with a probability of 4.438e-01 and stops, due to threshold exceeding issue. However, the optimal solution $\\Gamma\_S$ would be subset [4.438e-01, 1.027e-01, 3.320e-02, 1.804e-04, 5.383e-06, 5.2056e-07, 3.760e-08, 1.521e-12] with an entropy of 0.6777 and a retained mass of 0.580 > 0.4438 (greedy solution). Thus, there are special cases where the optimal solution to ECMM would prefer a token with a lower probability over a token with a higher probability.

Exaggerated claim(s): "solving the ECMD problem can ideally balance creativity and coherence"

Based on the above example, we argue that solving the ECMM problem can yield solutions better than the greedy algorithm, particularly for corner cases. This rationale also helps ECMM maintain a significantly better creativity-coherence trade-off over the SoTA: maximizing $\\Gamma\_S$ enforces coherence, while the entropy cap gates creativity and prevents coherence degradation. Top-H implements this balance greedily: it locks in the most probable tokens first, then expands until the entropy budget is about to be exceeded.

The Choice of JSD:

We chose JSD because it is a standard, bounded, and symmetric measure of divergence. Crucially, as we prove in Theorem 1, minimizing the JSD between our distributions is mathematically equivalent to maximizing the retained probability mass ($\\Gamma\_S$​). This is a powerful and intuitive result: our theoretically grounded objective of minimizing divergence simplifies to the very practical goal of keeping the most probable tokens, which directly addresses the coherence objective.

Vacuous theoretical results (Theorem 3):

We agree that mere termination on a finite vocabulary is trivial. Theorem 3 is not about finiteness, rather establishes a strict monotonicity property of entropy under probability‑ordered inclusion. Thus, there is a single crossing point of the constraint $H(q_k)\le \alpha H(p)$ that greedy algorithm finds. This ensures: 1. Strict monotonicity guaranteeing a unique stopping index. Without it, we could overshoot/undershoot the entropy cap. 2. The theorem is the formal link between the entropy constraint and the greedy prefix growth, making it an inevitable consequence—not just a heuristic. 3. The ordering assumption is what turns an NP‑hard subset problem into a linear‑time scan.

Alternative theoretical foundation:

We agree that a maximum‑entropy (ME) view over confidence‑ordered prefixes is intuitive and top‑H can be stated that way. However, ECMM formulation is strictly more general (it does not assume contiguity), yields the divergence–mass equivalence (Thm. 1), and provides the optimality/complexity guarantees that the ME‑prefix view alone cannot.

1. Maximum-Entropy restatement of Top-H (under prefix restriction)

Let $p$ be sorted by probability: $p_{(1)} \ge p_{(2)} \ge \dots$. For the prefix $S_k = \{1,\dots,k\}$, define $q_k(i) = \frac{p_{(i)}}{\sum_{j \le k} p_{(j)}} \quad (i \le k).$

If we restrict ourselves to such prefixes, then choosing “the most uninformed (highest entropy) distribution that still respects the entropy cap” reduces to:

$k^\star = \max\{k \mid H(q_k) \le \alpha H(p)\}.$

Since $H(q_k)$ increases monotonically with $k$ when tokens are added in descending $p_{(i)}$ (Theorem 3), this rule is identical to Top-H that adding tokens until the entropy cap would be violated. Thus, under prefix restriction, the top-H is equivalent to the maximum-entropy set (subject to the $\alpha H(p)$ threshold) over the confidence-ordered prefix.

2. Relation to ECMD/ECMM

ECMD objective starts from minimizing divergence of $p$ (fidelity) while bounding randomness. We show that minimizing JSD under the same entropy cap is equivalent to maximizing retained probability mass $\Gamma_S = \sum_{i \in S} p_i$ (ECMM). Over ordered prefixes, $\Gamma_{S_k}$ is monotone in $k$, so the ECMM optimum picks the same $k^\star$.

Thus, the ME-prefix view rephrases the same greedy rule—but only after imposing the prefix constraint.

Why retain ECMD as the main formulation?

1. No prefix assumption: ECMD does not assume contiguity. The fact that greedy prefixes are (near-)optimal is derived, not assumed.
2. Theoretical guarantees: ECMD/ECMM yields (i) the divergence–mass equivalence (Theorem. 1) and (ii) NP‑hardness of the unrestricted problem—explaining why a greedy approximation is appropriate.
3. Interpretability & control: The entropy ratio $\alpha$ gives a clear knob linking creativity (entropy) to fidelity (mass), independent of the prefix framing.
4. Unification: The ECMD lens naturally relates Top‑H to other decoding schemes (temperature, nucleus) within one objective-driven framework.

Dear Reviewer,

Thank you for the time and effort you’ve put into reviewing our submission. We’ve carefully considered each of your comments and concerns, and have done our best to address them thoroughly in our rebuttal with the necessary theoretical or empirical evidences.

We understand that time-bound reviewing process is demanding. However, as the deadline of discussion is approaching soon, we would greatly appreciate it if you could take a moment to revisit our response and, if possible, review the scores. With the enhanced thoroughness driven by your feedback, we believe top-H decoding has emerged as a promising alternative to the existing approaches.

Your feedback is invaluable and can help ensure that our clarifications are appropriately considered in the final decision.

Please don’t hesitate to reach us out in case any further discussion is needed.

Best regards,

Authors

Clarification on Top-n = 100:

The use of top-n = 100 is an empirical preprocessing step aimed at improving computational efficiency by avoiding the processing of tokens with zero probability, which do not contribute to the entropy calculation. In all of our experiments, tokens ranked beyond the top 100 consistently have zero probability in the sorted token distribution. Therefore, truncating at this threshold allows us to safely discard these tokens without affecting the results. We would like to emphasize that this is purely an implementation detail and does not impact any of the theoretical arguments or conclusions presented in the paper.

We would thus hope the reviewer to reassess their evaluations based on both the merit of the problem formulation followed by a practically viable solution, as we truly believe the draft's initial derivation is essential for the community to have a clear picture on the sampling problem, which has been largely missing in literature.

Regards,

Authors

We thank the reviewer 2JSm for their response to the rebuttal, and happy to note that some parts of the response were convincing. We now further clarify on the remaining.

Clarity to the Human eval setup:

We would like to clarify the rationale behind our human evaluation setup. Our design closely follows the min-p paper (accepted as ICLR 2025 Oral). This choice was made, primarily to ensure a fair and direct comparison with min-p, which represents the current state of the art. Due to the limited time available during the rebuttal phase, we restricted the number of outputs per configuration to three. Increasing this number would have significantly increased the burden on participants and was not feasible within the limited timeline. However, we are committed to conducting a more comprehensive human evaluation with a larger set of samples for the camera-ready version of the paper, should it be accepted.

Clarification on when is "dropping high confidence tokens is good"?

As we highlighted in the earlier responses, despite it may seem counterintuitive, dropping the highest-probability token can improve overall generation quality in certain settings. This happens when we care more about maintaining diversity and coherence across multiple plausible options rather than simply following the single most likely path. Following are few examples:

a) Open-ended generation (e.g., storytelling, ideation): The model faces multiple valid directions for the next word—like the first sentence of a story or brainstorming ideas. The top token may dominate due to a small margin, but excluding it allows several alternative directions to enter the beam, giving the decoder a richer set of continuations.

b) Multi-modal or ambiguous contexts: The probability mass is spread across distinct semantic modes (e.g., “cat”, “dog”, “rabbit” are all plausible continuations). Greedy will lock into just one mode (e.g., “cat”), suppressing others. ECMM allows each cluster to be represented by a token.

c) High entropy contexts: When the model is uncertain (i.e., the distribution is flat in temperature scaling), dropping one large token frees up entropy space and results in outputs that are more faithful to the overall model uncertainty rather than dominated by noise from a single strong guess.

d) Filtering out “boring” or unhelpful dominant tokens: In local sampling, the single highest-probability token is often a structural or low-content token—punctuation, whitespace/“space” tokens, quotes, braces, or function words (“the”, “to”). These tokens are important for grammar, but they add little semantic variety. When high creativity is desired, letting one such token monopolize the shortlist crowds out several content-bearing alternatives. On the hand, dropping such a token can free up entropy budget for several semantically rich alternatives that drive the story, reasoning, or creativity forward.

On the unnecessary complication of ECMM:

While the ECMM formulation may seem "unnecessary and complicated", we would emphasize on our goal here: to demonstrate the sampling problem's relevance as the entropy maximization problem, which is essential in understanding the theoretical underpinnings of sampling diversity and optimal token distribution. Additionally, we would respectfully differ on the claim that "the complexity of the ECMM formulation would be unnecessary as the Greedy version is not the exact solution." There are two points to consider: firstly, the ECMM is a more accurate—albeit NP-hard—formulation that yields a principled perspective on the objective landscape and helps frame the underlying optimization problem rigorously, which we believe is essential in terms of understanding the problem in its core relevant format. Secondly, Greedy, on the other hand, acts as an efficient approximation to this formulation. Neither of these perspectives has been previously explored in the literature in this context. We thus strongly believe the problem and its proposed approximate and viable solutions merit the community's attention, particularly in light of our newly evaluated empirical results and theoretical insights. Nevertheless, we resonate with the reviewer’s broader philosophy, and would like to emphasize that, "despite the theoretical establishment (which we believe is essential for the community to gain a comprehensive understanding), we also provide a practically viable approximate alternative that currently achieves state-of-the-art performance." Demonstrating the Greedy approach only would have left the work under-motivated and lacking theoretical grounding, limiting its generalizability and long-term value for the community to investigate.

We thank the reviewer ELnF for the insightful feedback and clarify their concerns below.

Justification for constraint form:

The choice of the entropy constraint form, $H(q) \le \alpha H(p)$, is central to our method's theoretical foundation and is chosen for its ability to dynamically balance creativity and coherence. We aim to develop a sampling framework explicitly aware of the model's confidence, not just based on the single highest-probability token, but on the overall shape of the probability distribution. Shannon entropy $H(p)$, is a measure for the uncertainty of the entire distribution $p$. Additionally, the formulation establishes an adaptive upper bound on the randomness of the selected token distribution, $q$. This way the generation can retain the following:

Adapts to Model Confidence: The constraint is proportional to the entropy of the model's original prediction, instead of a fixed value. Thus, when the model is uncertain (the distribution is flat, with a high $H(p)$), it allows the sampling distribution $q$ to have a higher entropy, encouraging exploration and creativity when multiple tokens are plausible candidates. Alternately, when the model is confident (distribution with sharp peaks, with a low $H(p)$), it forces the sampling distribution $q$ to have a lower entropy, preserving coherence by restricting the sampling pool to high-probability tokens.

Provides Control: The hyperparameter $\alpha \in$ (0,1) acts as a knob to control the trade-off between creativity and coherence.

Exploration of alternative formulations:

We considered approaches including Mirostat [1],min-p [2], and top-p [3]. However, these approaches either fail to adapt to the model's confidence or lack a direct theoretical link to the distribution's overall uncertainty.

Incremental contribution over $\eta$‑sampling

Despite both works control randomness via entropy, following are key differences: $\eta$‑sampling specifies a rule (“clip until entropy < $\eta$”) without any optimization to balance between creativity and coherence while ECMD starts from first principles, minimize divergence from the model distribution subject to an entropy cap. We further demonstrate the divergence-mass equivalence for the first time with an information-theoretic justification. We analyze the unrestricted subset selection problem (no contiguity imposed on the order of token selection) and show it is NP-hard. This explains why a naive global optimizer is impractical motivating our greedy restriction. We further prove monotone entropy growth (Thm. 3) within probability-ordered prefixes giving a unique stopping point and justifying Top‑H. $\eta$‑sampling offers neither hardness results nor structural theorems. ECMD provide a single objective that subsumes and relates diverse decoding heuristics (T, nucleus, top‑k) under clear trade-off parameters $\alpha$. We quantify how close the greedy solution is to the true optimum (Fig. 5: conservative mean − $\sigma$ mass ratio ≥ 96%), while prior work lack these approximation results.

Approximation guarantee:

Please refer to the Discussion on formal approximation bound for the Top-H response to reviewer bsFC on this.

Computational overhead and timing comparisons:

Table R1T1 provides the comparison of the average runtime per token (ms/token) of Top-H with Top-p and Min-p. We used LLaMA3.1-8B-Instruct, LLaMA3.3-70B-Instruct, and Phi-3-Mini-3.8B. For each setting, we averaged results over 100 prompts from the AlpacaEval, generating 128 tokens per prompt.

R1T1

Specifically, we observe a negligible overhead of as low as 0.8%.

Complexity analysis: All methods begin with a sorting step, typically $\mathcal{O}(n ~ log ~ n)$. Top-p and Min-p performing CUMSUM and thresholding, respectively, both having a complexity of $\mathcal{O}(n)$. Below, we explain the incremental entropy accumulation of Top-H to be $\mathcal{O}(n)$. The Theorem 3 of the paper leads to the following algorithm for computing entropy incrementally:
Let us define $h_{j-1} = \sum_{i=1}^{j-1} p_i \log p_i$, so the expression becomes: $H(q^{j-1}) = \log (\Gamma_{j-1}) - \frac{h_{j-1}}{\Gamma_{j-1}}$
Now, incremental entropy accumulation pseudocode:

Initialize Γ ← 0,  h ← 0,  H ← 0
for each step j:
    Γ ← Γ + p_j
    h ← h + p_j log p_j
    H ← log(Γ) - h / Γ

This runs in $\mathcal{O}(n)$ time.

Need for Human Eval:

We have now conducted Human Eval of LLM-generated texts using a setup similar to that of min-p, and compared with Top-p and Min-p. We recruited 14 PhD students for this. We used LLaMA3.1-8B-Instruct with texts generated using a prompt adapted from the min-p framework: “Write me a creative story.” For each configuration, we generated three outputs, capped at 512 tokens. Participants were asked to evaluate them along quality and diversity with rating on a scale of 1-10. Results are shown in R1T2.

R1T2

Hyperparameter robustness:

Please note that Figure 4 of the draft shows the creativity-coherence trade-off across a range of $\alpha$ values. Specifically, the changes in coherence and creativity metrics are gradual with changes in $\alpha$, indicating it to not be overly sensitive to small adjustments. Intuitively, $\alpha$ controls the balance between confidence and diversity in the decoding, with small $\alpha$ encouraging more deterministic outputs while its large value increasing the randomness and thus encouraging creativity. This controllability gives users practical guidance: moderate $\alpha$ values (around 0.4) offer a balanced trade-off, and users can adapt $\alpha$ based on their preference for coherence versus creativity. Additionally, the following table R1T3 reports GPQA accuracy and the average sampling pool size across different $\alpha$ values. The experiment is done using LLaMA3.1-8B-Instruct model with temperature T = 1.5. These results show that: 1. Larger $\alpha$ values slightly reduce accuracy, which aligns with the nature of GPQA’s graduate-level questions that benefit from more confident (less diverse) answers. 2. Sampling pool size increases with $\alpha$, providing more generative options and supporting the creativity aspect observed in Figure 4.

R1T3

Importantly, both accuracy and sampling pool size show smooth and consistent trends, reinforcing the method’s robust behavior. For moderate $\alpha$, accuracy changes smoothly; the small fluctuations at higher $\alpha$ are expected, as higher $\alpha$ increases the sampling pool and thus randomness.

---

References

[1] Mirostat: A neural text decoding algorithm that directly controls perplexity, arXiv.

[2] Turning up the heat: Min-p sampling for creative and coherent llm outputs, arXiv .

[3] The curious case of neural text degeneration, arXiv .

Dear Reviewer,

Thank you for the time and effort you’ve put into reviewing our submission. We’ve carefully considered each of your comments and concerns, and have done our best to address them thoroughly in our rebuttal with the necessary theoretical or empirical evidences.

We understand that time-bound reviewing process is demanding. However, as the deadline of discussion is approaching soon, we would greatly appreciate it if you could take a moment to revisit our response. With the enhanced thoroughness driven by your feedback, we believe top-H decoding has emerged as a promising alternative to the existing approaches.

Your feedback is invaluable and can help ensure that our clarifications are appropriately considered in the final decision.

Please don’t hesitate to reach us out in case any further discussion is needed.

Best regards,

Authors

Dear reviewer ELnF,

We remain thankful to your insightful reviewing comments. We provided our best attempt to address your remaining concerns with necessary evidences in the rebuttal. As the "author-reviewer discussion period" is ending in few hours, we hope you to provide your kind consideration to incorporate the substantial rebuttal evidences, that we believe has significantly strengthened the contributions.

Best

Authors