Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression

[W1, Mixed-Precision Comparison Fairness]

We agree that a fair evaluation of GLVQ’s core contribution, group-specific lattice quantization, should be based on uniform-precision settings, independent of our salience-driven bit allocation (SDBA) strategy.
To this end, uniform-precision variants of GLVQ (denoted GLVQ-u or GLVQ w/o bit allocation) were already included in our original submission and reported in Appendix A.4. For clarity and ease of comparison, we reproduce the key results below.

Table 1. Results of GLVQ-u.

As the table shows, even without salience-driven bit allocation, GLVQ-8D-u improves perplexity over the current SOTA PTQ method QTIP. This confirms that the performance gains stem from the underlying group-specific lattice design, rather than auxiliary heuristics such as mixed-precision allocation.

We will clarify this discussion in the revised paper and add a footnote directing readers to Appendix A.4 for the full set of uniform-precision ablation results.

[W2, Limited Evaluation Scope]

We agree that more challenging tasks such as MMLU provide a stronger test of generalization under quantization. To this end, we evaluated GLVQ on the MMLU benchmark for both Qwen-7B-chat and Qwen-14B-chat models, using standard 5-shot accuracy.

Table 2. MMLU accuracy (%) with different methods. Higher is better.

As shown in the table, GLVQ yields significantly higher MMLU accuracy than GPTQ at both 2-bit and 3-bit settings. In particular, at 2-bit precision, GLVQ improves over GPTQ by +22 points on Qwen-7B-chat and +29 points on Qwen-14B-chat. Even at 3-bit, GLVQ achieves consistent gains over GPTQ while remaining close to the BF16 reference.

These findings demonstrate that GLVQ maintains abstract reasoning and factual recall better than prior methods even on structurally different models like Qwen, highlighting its robustness across architectures.
We will incorporate this table and discussion into the revised manuscript to provide a broader and more rigorous evaluation.

[Q1, Effectiveness and Efficiency of Companding]

1. Computational cost

The $\mu$-law requires only a fused exp or LUT lookup per weight, while RHT/FFT introduce an extra $128 \times 128$ orthogonal multiply per group. This results in a 2% vs. 9–11% throughput drop, respectively.

2. Quantization fidelity (2-bit uniform, LLaMA-2-7B)

Table 3. Ablation of incoherence processing strategies.

$\mu$-law yields a larger perplexity drop than RHT (–0.59 vs. –0.40) by normalizing heavy-tailed weights before lattice quantization. Combining both gives a modest gain (–0.65), suggesting partially overlapping effects. In terms of efficiency, $\mu$-law costs only 2% throughput loss, whereas RHT introduces a heavier 9% drop due to its $\mathcal{O}(n \log n)$ transform, making $\mu$-law preferable for GPU and edge deployment.

The group-wise $\mu$-law companding offers a better fidelity–latency trade-off than RHT or FFT: lower perplexity at minimal compute overhead. While RHT can be layered for slight improvements, it is not essential to GLVQ’s performance. We will include this table and the above analysis in the revised manuscript.

[Q2, Decoding Efficiency Comparison]

Key implementation choices that minimize decoding overhead

GLVQ achieves competitive decoding speeds despite its arithmetic-based design by leveraging several carefully engineered optimizations:

1. On-the-fly fusion inside the GEMM kernel.
   Both the inverse $\mu$-law transformation and the matrix multiplication $G_g \mathbf{z}$ are fused into the fragment loading stage of the Tensor Core WMMA pipeline. These operations are performed on 16-value int2 register tiles, introducing no additional global memory traffic and incurring negligible latency due to full overlap with the GEMM execution.

2. Register-resident constant matrices.
   Each $G_g \in \mathbb{R}^{8 \times 8}$ consists of only 64 FP16 values (0.125 kB), which is smaller than a single Tensor Core fragment (0.256 kB). As a result, CUDA retains $G_g$ in registers across the warp, completely avoiding L1/L2 cache accesses.

To quantify the impact of these optimizations, we conducted the following micro-benchmark on an NVIDIA A100 GPU using CUDA 12.1:

Table 4. Micro-benchmark: 128×4096 GEMM at 2-bit precision.

The additional 0.09 µs decoding overhead introduced by GLVQ is negligible compared to the total GEMM time of ~135 µs. In end-to-end inference, this results in less than a 3 tokens/s throughput difference (see Table 3), while achieving substantially improved quantization fidelity.

Why arithmetic decoding can outperform LUT-based methods on modern GPUs

• Compute-bound regime.
  On Ampere GPUs, each byte of L2 bandwidth supports approximately 2 FP16 FMAs. GLVQ leverages ALUs and register space for decoding, whereas LUT-based methods rely on memory access, which becomes a bottleneck in compute-bound settings.

• Warp-level parameter reuse.
  Since each $G_g$ and $\mu_g$ is shared across all threads within a warp, GLVQ avoids thread divergence and enables efficient, fully parallelized decoding.

In summary, thanks to aggressive kernel fusion and register-level optimization, GLVQ achieves decoding latency within 0.09 µs of QuIP#, translating to under 3% throughput difference, while delivering superior quantization fidelity. We will incorporate these engineering insights and benchmarking details into the revised manuscript to clarify decoding efficiency.

[Q3, Throughput Discrepancy with QuIP#]

Clarification
The throughput discrepancy arises from two distinct configurations reported in the original QuIP# paper. Specifically:

• Table 5 reports 170.5 TOK/s using an optimized stack with FlashAttention 2 and Triton-based custom kernels.
• Table 6, by contrast, reports 106.7 TOK/s using the standard Hugging Face library; this is the same stack used in GLVQ.

To ensure fairness and consistency, our Table 3 adopts the Hugging Face inference configuration across all methods. We therefore report the 106.3 TOK/s figure, matching Table 6 of the QuIP# paper.

To verify that this choice does not alter the relative comparison, we also benchmarked both QuIP# and GLVQ under FlashAttention 2. As shown in Table 1, the relative gap between QuIP# and GLVQ remains stable (approximately 6–7%) under both inference engines.

Table 5. Throughput of QuIP# under different inference engines (LLaMA-2-7B, 2-bit uniform).

[W1 & W2, Issue with evaluation procedure]

First, We thank the reviewer for catching citation discrepancy. The reviewer is correct: in the original manuscript, we inadvertently cited FP16 baseline results for Llama-2 under a 2k context length instead of the correct 4k context used during evaluation. We have now corrected this in the revised version. Importantly, we emphasize that this correction does not affect the results or conclusions of our method (GLVQ).

Besides, it is indeed uncommon for a post-training quantized (PTQ) model to outperform its original FP16 baseline. However, such outcomes are not unprecedented and can occur under certain conditions, as observed in prior work. For example, Table~6 in the QTIP paper (NeurIPS’24) shows that several 4-bit quantized LLaMA checkpoints achieve higher zero-shot accuracy than their FP16 counterparts. This has been attributed to mild regularization effects from quantization, or improved alignment between quantizer and inference distribution.

Fine-tuning explains small improvements.
In our GLVQ pipeline, a lightweight fine-tuning step is optionally performed after quantization. This step fine-tunes only the group-specific lattice generation matrices $G_g$ and companding parameters $\mu_g$, using a small external corpus of ~8M tokens. This fine-tuning allows the quantizer to better adapt to actual inference-time statistics and introduces a regularization-like effect, sometimes yielding perplexity slightly better than FP16.

GLVQ without fine-tuning still outperforms.
To ensure that our main gains are not solely due to the optional refinement, we also evaluate a pure PTQ variant of our method, named GLVQ-w/o-ft, where no fine-tuning is applied post-quantization. The table below shows that GLVQ-w/o-ft still consistently outperforms recent state-of-the-art PTQ methods at 2–4 bits precision.

Table 1. GLVQ results with and without fine-tuning on WikiText2.

[W3, New results on Llama-3]

We have reproduced our pipeline on Llama‑3‑8B and Llama‑3‑70B under the same 2–4 bit settings on WikiText2. Results are summarized below and will be added to the revised paper.

Table 2. Perplexity ↓ results on WikiText2 of Llama‑3.

[W5, Speed-up numbers for AQLM]

Our Table 3 cited the value 20.6 TOK/s for “AQLM (2‑bit, LLaMA‑2‑7B, RTX 4090)” based on Table 7 of the QTIP arXiv v1 paper (page 8), which referred to the same experimental setup. However, the original AQLM paper (arXiv v1) did not report throughput; this was added later in arXiv v3.

In arXiv v3, Table 14 reports two throughput numbers on RTX 3090: 65.3 TOK/s and 114.1 TOK/s. The latter corresponds to a faster variant using a different kernel, but the 65.3 TOK/s figure matches the exact configuration used to report perplexity and accuracy. Therefore, the 65.3 number is the appropriate one for comparison under consistent conditions.

To resolve this ambiguity, we have re-run the official AQLM code on our own RTX 4090 setup using the same batch size and context length (batch = 1, sequence length = 2048) as all other methods in Table 3. Under this unified setting, AQLM achieves 91.7 TOK/s.

[Q1, Output error]

We would like to offer the following points to clarify GLVQ’s relationship to the information-theoretic lower bound:

First, GLVQ is explicitly designed to reduce group-wise reconstruction error through the use of learnable lattices and companding parameters. This aligns closely with QTIP’s goal of minimizing layer-wise distortion, though our method achieves this via group-specific vector quantization rather than Hessian-weighted scalar quantization.

Second, although we do not report NMSE explicitly, GLVQ consistently achieves better perplexity than QTIP in all 2-bit settings. Since previous works (including QTIP) have established a strong empirical correlation between NMSE and perplexity, this improvement indirectly suggests that GLVQ attains comparable or lower reconstruction error at the output level.

[Q2, Lattice quantization]

Indeed, given a full‑rank generation matrix $G_g \in \mathbb{R}^{d \times d}$ and Babai rounding, GLVQ performs a scalar quantization in the latent orthonormal basis $G_g^{-1}$.
However, two properties make this seemingly simple scheme highly expressive in practice:

1. Learnable affine map.
   By optimising $G_g$ per group we can
   (a) rotate the data so that major covariance directions align with lattice axes, and
   (b) scale each axis independently.
   The resulting Voronoi cell is a parallelotope whose orientation and aspect ratio are free parameters—already a rich family covering the optimal vector quantizers for any elliptically contoured distribution (Whitened Gaussian, Laplacian, etc.).

2. Non‑linear companding.
   We further learn a monotone companding function $F_{\mu_g}$ that redistributes probability mass towards the centre of the lattice. The overall mapping:

   $$\mathbf{w} \xrightarrow{F_{\mu_g}} \mathbf{y} \xrightarrow{G_g^{-1}} \tilde{\mathbf{y}} \xrightarrow{\text{round}} \mathbf{z}$$

   is piece‑wise affine; its pre‑images can approximate arbitrarily skewed or heavy‑tailed densities with vanishing codebook overhead (the lattice is infinite).

Why not arbitrary VQ?
Free‑form vector quantization can in theory tile any convex region, but comes with exponential codebook growth ($\sim 2^{bd}$ entries), $O(d)$ lookup latency, and large table storage.
GLVQ retains $O(d^2)$ decode cost and $O(d^2)$ parameters per group, independent of bitrate, while still matching the local covariance structure; empirically this captures the vast majority of quantization gain, as Table 2 shows GLVQ within $0.08$ PPL of VQ‑LLM (free codebook) yet decoding $4.8\times$ faster.

Formal perspective.
Given any continuous distribution with covariance $\Sigma$, there exists a scaling matrix $A$ such that:

Choosing $G_g = A^{-1}$ reduces the optimum $d$‑dimensional VQ problem to axis‑aligned scalar quantization without increasing distortion (see Gersho & Gray, Ch. 6).
Thus, lattice + Babai is universally rate‑distortion optimal up to a constant gap while remaining computationally tractable.

[Q3, Inference framework]

Hugging face. Context length 2048.

[Q5, overhead caused by mu-law transformation]

Thank you for raising this important point. While the $\mu$-law inverse transform $F^{-1}_{\mu_g}$ is indeed non-linear, its practical overhead is negligible due to (1) its limited scope, and (2) its tight integration with GPU-friendly fused kernels.

First, $F_{\mu_g}$ is only applied during quantization time. At inference time, only the inverse mapping is used to decode the dequantized weights just before computing the matrix–vector product.

In GLVQ, weights are decoded group-by-group, and each group corresponds to a matrix of shape $4096 \times 128$ (i.e., one transformer layer's weight tile). Therefore, each application of $F^{-1}_{\mu_g}$ processes exactly $4096 \times 128 = 524{,}288$ FP16 numbers. This decoding is executed once per group, after which the full group matrix is reused in multiple GEMV calls.

To assess the actual cost, we conducted timing measurements on LLaMA‑2‑7B at 2‑bit precision using an RTX 4090. The table below shows the result of ablation by replacing $F^{-1}_{\mu}$ with the identity function.

Table 3. Impact of $\mu$-law inverse on decoding throughput (LLaMA‑2‑7B, 2-bit, RTX 4090).

[Q1, Training GPU hours]

We measured wall-clock quantization time using a single NVIDIA A100 80GB under identical software environments for all methods: GLVQ, GPTQ, QuIP#, and AQLM.

Table 1. End-to-end quantization time on a single A100 GPU.

GLVQ completes 70B quantization in approximately one day on a single A100. While this is slower than GPTQ and QuIP#, it is over 10× faster than AQLM. The added cost mainly comes from optimizing small (8×8) lattice transformations for each weight group, which are trivially parallelizable.

This one-time conversion cost yields strong returns: at 2-bit, GLVQ improves WikiText2 perplexity by 0.6–1.1 points compared to GPTQ and AWQ, while maintaining comparable throughput. Since quantization is performed offline, these accuracy gains benefit all downstream inference.

The method scales efficiently. When run on 8 A100s, GLVQ quantizes LLaMA‑2‑7B in just 25 minutes, enabling practical deployment for large models. Furthermore, reducing the number of lattice iterations from 12 to 4 can shorten quantization time by up to 40% with only a small accuracy drop (typically 0.05–0.07 perplexity), providing a useful knob for compute-constrained settings.

We will include the new results and this discussion in the revised paper to clarify the time–accuracy tradeoff of GLVQ.

[Q2, Baseline Comparisons]

We thank the reviewer for highlighting recent methods such as OmniQuant and CBQ.

Our primary focus in this work is to evaluate post-training vector quantization (VQ) methods. Accordingly, we benchmarked GLVQ against representative VQ-based baselines, AQLM, QuIP#, and QTIP, which adopt codebook-based, groupwise quantization strategies.

OmniQuant was already included in our original experiments, and its results are reported in Table 1 of the main paper. CBQ, on the other hand, performs non-uniform scalar quantization with cross-block error correction. While originally considered orthogonal to our focus on VQ methods, we appreciate the reviewer’s suggestion and have conducted additional experiments to compare GLVQ with CBQ at 2-bit and 4-bit settings under the same evaluation protocol (WikiText2, LLaMA‑1‑30B and 65B).

Table 2. Comparison with CBQ on WikiText2 (LLaMA‑1).

As shown in Table 1, GLVQ consistently achieves lower perplexity than CBQ across both model sizes and bitwidths. These additional results will be incorporated into the revised version of the paper, together with a brief discussion. We thank the reviewer again for encouraging a more comprehensive comparison.

[Q3, Error Bars]

We appreciate the reviewer’s suggestion and fully agree that reporting confidence intervals would strengthen the empirical claims.

Due to time constraints, we were unable to re-run all experiments across multiple seeds before the rebuttal deadline. However, we are committed to including 95% confidence intervals (based on five seeds) for all key perplexity and accuracy results in the final version of the paper and supplementary material.

A note will also be added to clarify that all reported numbers are averages over multiple seeds with corresponding confidence intervals. Thank you for encouraging stronger statistical rigor.

[Q4, Activation Quantization]

We thank the reviewer for this insightful question. Extending GLVQ to quantize activations and KV-cache is a natural direction. While feasible in principle, several practical considerations must be addressed:

First, activation and KV-cache statistics are input-dependent and non-stationary, unlike model weights. Designing a single static lattice per layer may be suboptimal unless proper normalization or dynamic rescaling is introduced. This challenge can be mitigated using techniques like SmoothQuant [@smoothquant], which apply lightweight affine transformations to stabilize activation distributions layer-wise. Once stabilized, a per-layer lattice codebook becomes viable.

Second, runtime efficiency is critical. Since activations are generated and consumed on-the-fly, any decoding operation must match the low-latency constraints of inference. Fortunately, lattice decoding with small dimensions (e.g., $d = 4$) is fast and can be further accelerated using SIMD instructions or table lookups. For KV-cache blocks that are reused across many time steps, the amortized cost of decoding is even lower, making lattice compression particularly attractive in that setting.

Lastly, memory reuse patterns and GPU bandwidth constraints need careful evaluation. Compression benefits will only be realized if the decoding overhead is small relative to the memory bandwidth saved. To address this, we envision using compact LUT-based inverse $\mu$-law decoding to reduce runtime cost further.

In summary, while activation and KV-cache quantization introduces new challenges, we believe GLVQ can be extended to this setting through (i) per-layer rescaling, (ii) efficient low-dimensional decoding, and (iii) amortized or LUT-based companding. We are actively exploring these directions in future work.

[Q5, Hardware Efficiency]

In this submission, we focused primarily on evaluating memory-bandwidth–limited throughput on server-class GPUs such as the RTX 4090 and A100. As such, we have not yet benchmarked wall-clock latency on mobile SoCs or measured energy consumption in terms of Joules per token.

That said, we fully agree that these metrics are essential for assessing deployment feasibility on edge devices. As a next step, we plan to port the CUDA kernels to mobile hardware platforms, including ARM Mali and Apple M-series chips via torch.metal_backend. We also intend to profile inference on Qualcomm Snapdragon 8 Gen 3 (Adreno 750) using the QTI AI SDK, which will allow us to collect both latency (ms/token) and energy (J/token) metrics under realistic mobile conditions.

These measurements require additional engineering work and optimization, particularly around kernel portability and runtime instrumentation. We plan to include the results in an extended technical report and will reference this in the camera-ready version of the paper.

We thank the reviewer again for pointing out this important direction for future evaluation.

[Q6, Ethical Safeguards]

We fully share the reviewer’s concern that more compact LLMs, while enhancing accessibility and deployment efficiency, may also lower the barrier for misuse, such as scalable disinformation or malicious automation.

In the camera-ready version, we will include a brief “Responsible Use and Limitations” subsection. This will highlight that a model card will be released alongside GLVQ, explicitly stating intended applications (e.g., on-device summarization, QA), out-of-scope uses (e.g., generation of deceptive or harmful content), and known limitations such as hallucination and toxicity.

We will also note that GLVQ preserves the alignment properties of the original FP16 model, including RLHF or instruction tuning, and caution against compressing unaligned checkpoints. Since GLVQ operates only on the weight tensor, existing watermarking methods (e.g., TreeRing) remain applicable. We will recommend practitioners verify watermark presence before deployment and, where feasible, use an external safety filter or moderation API in downstream applications.

We believe this addition provides appropriate ethical context and encourages responsible use of compressed LLMs.

[W1 & Q1, Comparison with NestQuant, VQ-LLM]

We thank the reviewer for highlighting recent lattice-based quantization works, including NestQuant and VQ-LLM. We respectfully clarify that these two papers play fundamentally different roles in the LLM quantization landscape.
(i) NestQuant is a lattice-based post-training quantization (PTQ) method that constructs nested lattice structures for compressing both weights and activations in LLMs. It is directly comparable to GLVQ in terms of quantization performance and perplexity.
(ii) VQ-LLM, in contrast, is not a quantization algorithm, but rather a high-performance kernel generation framework for accelerating inference on already vector-quantized models (e.g., those produced by QuIP#, CQ, or GPTVQ). It focuses on efficient dequantization and computation fusion, and does not propose a new quantization method.

In the revised version, we will include a discussion of both works to clarify their respective scopes. To enable a direct comparison with NestQuant, we evaluated GLVQ on Llama-2 models (7B, 13B, and 70B) under 4-bit quantization using the WikiText2 benchmark. The results, summarized in Table 1, show that GLVQ consistently improves perplexity over NestQuant by 0.3--0.5 points across all model sizes. These results demonstrate the effectiveness of GLVQ’s group-specific, learnable lattice design in achieving stronger quantization performance than fixed nested lattices at low bit-widths.

Table 1. Perplexity ↓ Comparison with NestQuant on WikiText2 (Llama-2).

[W2 & Q2, Fractional bit-widths and below 2 bits setting]

We appreciate the reviewer’s suggestion to evaluate GLVQ under fractional and ultra-low bit-width settings.

Fractional bit-widths are fully supported in our framework. While our main paper reports results at integer average budgets (e.g., 2, 3, 4 bits per weight) for comparability with prior work, GLVQ’s salience-driven bit allocation (SDBA) operates at the group level. Each group is assigned an integer bit-width $b_g \in \{1, 2, 3, 4\}$, and the global bit-rate is given by the average $\bar{b} = \frac{1}{G} \sum_{g=1}^G b_g$. This allows us to realize any fractional target $b_{\text{avg}}$ (e.g., 1.5) by mixing different group-wise precisions (e.g., 50% 1-bit and 50% 2-bit). Importantly, the design of our codebooks and decoding process is invariant to such mixtures and requires no modification.

New experiments at fractional and sub-2-bit settings. To address the reviewer’s concern, we conducted additional experiments at 1.5 and 1.0 average bits per weight, using the same protocol as in the main paper. Results on WikiText2 for Llama-2 models are shown below:

Table 2. Perplexity ↓ results of GLVQ and competing methods at fractional and sub-2-bit settings on WikiText2 (Llama‑2).

These results confirm that GLVQ naturally supports fractional and ultra-low bit-width quantization. At an average of 1.5 bits per weight, GLVQ achieves perplexities of 7.01 (7B), 6.11 (13B), and 4.99 (70B), consistently outperforming PV-Tuning and significantly surpassing PB-LLM by large margins. Notably, GLVQ approaches the performance of 2-bit baselines despite using substantially fewer bits.

Even at an aggressive 1.0 bit-per-weight budget, GLVQ remains competitive, achieving perplexities of 7.83 (7B), 7.59 (13B), and 6.11 (70B). This represents a clear improvement over prior 1-bit methods such as BiLLM and OneBit, and is on par with or better than PV-Tuning at similar bit rates. Compared to BiLLM, GLVQ reduces perplexity by over 24 points on 7B and over 10 points on 13B under similar bit budgets.

These findings highlight the robustness of GLVQ’s group-specific lattice design and salience-driven bit allocation strategy. Unlike methods that require calibration or fine-tuning, GLVQ achieves strong performance even in the ultra-low-bit regime, making it a compelling solution for extreme compression scenarios.

[Q3, Better figure configuration]

Thank you for pointing out the visual ambiguity in Fig.~1. In the current version, the three sub-matrices ("before companding," "after companding," and "lattice-quantized") are rendered using the same color scheme, which indeed makes it difficult to visually distinguish the effects of group-specific companding and group-specific lattice quantization.

Planned revision.
We will revise Fig.,1 into a more intuitive and visually informative pipeline illustration that clearly differentiates each transformation stage:

1. Original weights.
   The full weight matrix will be shown with distinct color hues for each group. Groups with heavier-tailed distributions will use darker shades to visually convey statistical diversity across groups.

2. Group-specific companding.
   Each group is processed through its corresponding companding function $F_{\mu_g}$. The output matrix will be recolored using a unified pastel palette, emphasizing that group-wise dynamic ranges have been normalized post-companding.

3. Group-specific lattice quantization.
   For each group, we will overlay its corresponding lattice grid, and include a small 2D inset that visualizes the shape of the learned generator matrix $G_g$, with axes scaled and rotated differently per group. The quantized outputs will be shown as discrete markers, clearly distinguishable from the continuous pre-quantized values.

This redesigned figure will leverage both color (distributional contrast, companding effect) and spatial structure (lattice overlay, generator insets) to clearly communicate the impact of group-specific companding and the subsequent lattice selection. We believe this will fully address the reviewer’s concern and significantly improve the figure’s explanatory clarity.

[Q4, knapsack problem]

At first glance, selecting per-group bit-widths $b_g$ under a global bit budget may resemble a discrete knapsack problem. However, our setting introduces additional structure that enables a highly efficient and closed-form solution:

1. We restrict each group’s bit-width to a small discrete set: $b_g \in \{N{-}1,\,N,\,N{+}1\}$, and enforce a symmetric constraint as defined in Eq.~10. This ensures the average bit-width remains exactly $N$.

2. Each group is associated with a scalar salience score $\mathcal{S}_g$, reflecting its relative importance. By sorting all $G$ groups by salience $O(G\log G)$, we obtain the optimal assignment directly: the top $k$ groups receive $N{+}1$ bits, the bottom $k$ groups receive $N{-}1$ bits, and the remaining groups receive $N$ bits. This selection mechanism follows the double-pointer scheme introduced in Slim-LLM.

Because the bit choices are limited and symmetrically constrained, the search space reduces to identifying a single threshold index, rather than solving a general 0--1 knapsack. The overall algorithm runs in $O(G\log G)$, is deterministic, and yields an allocation that is provably optimal under our model.

[Q5, Extreme corner cases GLVQ-2D & -4D]

We appreciate the reviewer’s suggestion. In the original submission, we focused on 8D (canonical $E_8$) and 32D (stacked $E_8$ blocks) because they (i) align with SIMD register widths on modern GPUs and (ii) empirically offered a strong trade-off between accuracy and speed. That said, we agree that including smaller lattice dimensions provides better insight into how dimensionality impacts performance.

We have now added evaluations at 2D and 4D under identical settings. Results are shown in Table 1.

Table 3. Perplexity ↓ of competing methods on WikiText2 for Llama-1 and Llama-2.

These results reveal a consistent trend: increasing the lattice dimension leads to better perplexity across all model sizes. Compared to QTIP, GLVQ-2D underperforms slightly, while GLVQ-4D achieves comparable accuracy. From 4D to 8D, perplexity improves noticeably, and the 32D variant provides further gains with diminishing returns.

Thank you very much for your very detailed rebuttal! I appreciate the time and thought you did put into this and your additional results and clarifications do raise the value of the publication significantly. As a result I will increase my overall rating by 1 point.