ZipAR: Parallel Autoregressive Image Generation through Spatial Locality

Thanks to the reviewer for the valuable comments.

Q1: Utilize additional metrics to fully evaluate the method. To address this concern, we have expanded our evaluation by assessing ZipAR's performance using multiple metrics, including VQAScore, Human Preference Score v2, ImageReward, and Aesthetic Score, across three models: LlamaGen-XL-512, Lumina-mGPT-768, and Lumina-mGPT-1024. The results presented below demonstrate that our method significantly improves generation efficiency with little impact on output quality across various benchmarks.

Q2: Show some qualitative comparisons of different window sizes. We have provided qualitative visualizations of different window sizes, as referred to Figures 1, 8 in the paper and Figures 9, 10 in the supplementary material. Since we can not update PDF version in the current phase, we will add more qualitative comparisons of different window sizes in the revised version.

Q3: The effectiveness of speculative decoding in identifying short local window sizes that lead to insufficient information. Let $p_s$ and $p_{s+1}$ be the token distributions for window sizes $s$ and $s+1$. As referred to Eq. 2 in the paper, the acceptance probability for a candidate $x_s \sim p_s$ is defined as

$$\alpha(x_s)= \min\left(1, \frac{p_{s+1}(x_s)}{p_s(x_s)}\right)$$

The expectation of the acceptance can be formulated as:

$$\mathbb{E}_{x \sim p_s}\left[\alpha(x_s)\right]$$ $$= \sum_x p_s(x) \min(1, \frac{p_{s+1}(x)}{p_s(x)}) = \sum_x \min(p_s(x), p_{s+1}(x))$$

Theorem 1 (Relationship between Pairwise Minimum and Total Variation).
Let $p$ and $q$ be two probability distributions over the same discrete support $\mathcal{X}$. The sum of their element-wise minima satisfies:

$$\sum_{x \in \mathcal{X}} \min(p(x), q(x)) = 1 - \text{TV}(p, q),$$

where $\text{TV}(p, q) = \frac{1}{2} \|p - q\|_1$ is the total variation distance between $p$ and $q$.

Proof

For any $x \in \mathcal{X}$, observe that:

$$\max(p(x), q(x)) + \min(p(x), q(x)) = p(x) + q(x).$$

Summing over all $x$ yields:

$$\sum_{x} \max(p(x), q(x)) + \sum_{x} \min(p(x), q(x)) = \sum_{x} p(x) + \sum_{x} q(x) = 2. \quad (1)$$

By definition of the L1-norm, we have:

$$\|p - q\|_1$$ $$= \sum_{x} \|p(x) - q(x)\|$$ $$= \sum_{x} \max(p(x), q(x)) - \sum_{x} \min(p(x), q(x)). \quad (2)$$

Let $S_{\min} = \sum_{x} \min(p(x), q(x))$ and $S_{\max} = \sum_{x} \max(p(x), q(x))$. Then Equation (1) can be reformulated as:

$$S_{\max} = 2 - S_{\min}. \quad (3)$$

By substituting Equation (3) into (2), we have:

$$\|p - q\|_1$$ $$= (2 - S_{\min}) - S_{\min}$$ $$= 2 - 2S_{\min}.$$

Rearranging and using $\text{TV}(p, q) = \frac{1}{2} \|p - q\|_1$:

$$S_{\min} = 1 - \text{TV}(p, q). \quad$$

Therefore, the expected acceptance rate is formulated as:

$$\mathbb{E}_{x \sim p_s}\left[\alpha(x_s)\right] =$$ $$\sum_x \min(p_s(x), p_{s+1}(x))=1-\text{TV}(p_{s+1},p_s) \quad (3)$$

If $s$ is insufficient, the distributions $p_s$ and $p_{s+1}$ diverge significantly, implying $\text{TV}(p_s, p_{s+1}) \geq \Delta$ for a threshold $\Delta > 0$. By (3), the expected acceptance rate is upper bounded by:

$$\mathbb{E}[\alpha] \leq 1 - \Delta.$$

A low acceptance rate ($\leq 1 - \Delta$) prompts the algorithm to increase the window size to $s+1$.

If $s$ is sufficient, $p_s$ and $p_{s+1}$ are statistically indistinguishable ($\text{TV}(p_s, p_{s+1}) \approx 0$). By (3), the expected acceptance rate approaches 1:

$$\mathbb{E}[\alpha] \geq 1 - \epsilon \quad (\epsilon \approx 0),$$

allowing immediate sampling from $p_{s+1}$ without window expansion.

Thanks to the reviewer for the valuable comments.

Q1: The lack of experiments with 512 or higher resolution. For clarity, we would like to highlight that our experimental results already include higher-resolution evaluations, as referred to Table 2 in the paper. Specifically, the LlamaGen-XL model operates at 512x512 resolution and the Lumina-mGPT model operates at 768x768 resolution. To improve transparency, we will explicitly state these resolutions in the revised Table 2 caption. Moreover, to further address your point, we have conducted additional experiments using the Lumina-mGPT-1024 model at 1024x1024 resolution on various benchmarks. Due to the character limit here, please refer to Q1 in our response to Reviewer ELf9 for the evaluation results. These results demonstrate our method's effectiveness across multiple resolution scales.

Q2: Experiments are mainly implemented using the LlamaGen model. As referred to lines 299-300 in the paper, we integrate ZipAR with three state-of-the-art next-token AR visual generation models: LlamaGen, Lumina-mGPT and Emu3-Gen. Quantitative results can be found in Table 1-2 in the paper and the visualization results of these models can be found in Figures 1, 8 in the paper and Figures 9, 10 in the supplementary material.

Q3: Lack verification of other models, such as MAR. It should be noted that ZipAR is a training-free, plug-and-play parallel decoding framework for vanilla next-token AR visual generation models. However, MAR does not follow a next-token prediction generation paradigm.

Q4: Provide a brief pseudo code for the sampling process. Thanks for your valuable comment. We have provided a pseudo code for the sampling process, as shown below

# Pytorch-style Pseudo Code for ZipAR Sampling Process

# Image latent dimensions: H x W
# Minimum window size: s_min

# Initialize variables
total_columns = W                           # Number of columns
total_rows = H                              # Number of rows
decoding_rows = [0]                         # Rows actively being decoded
decoded_tokens = [[] for _ in range(H)]     # Decoded tokens for each row
pending_starts = []                         # Tentative new rows awaiting validation

while decoding_rows or pending_starts:
    # --- Step 1: Decode one token in each active row ---
    for row in decoding_rows:
        if len(decoded_tokens[row]) < total_columns:
            # Decode next token using AR model
            new_token = generate_token(row, len(decoded_tokens[row])) ## Generate token at position (row, len(decoded_tokens[row]))
            decoded_tokens[row].append(new_token)

    # --- Step 2: Process pending_rows
    new_pending = []
    for (new_row, old_token, old_prob) in pending_starts:
        new_prob, new_token = speculative_generate(new_row, 0) ## Tentative generation of token at position (new_row, 0)
        # Calculate acceptance probability
        r = uniform_sample()
        
        if r < min(1, new_prob[old_token] / old_prob[old_token]): ## Eq. 2 in the paper
            # Accept token and start decoding
            decoding_rows.append(new_row)
            decoded_tokens[new_row].append(new_token)
        else:
            # Resample from difference distribution
            resampled_prob, resampled_token = resample_distribution(new_prob, old_prob) ## Eq. 3 in the paper
            new_pending.append( (new_row, resampled_token, resampled_prob) )
            
    pending_starts = new_pending

    # --- Step 3: Check for completed s_min-1 tokens to initiate new rows ---
    for row in list(decoding_rows):
        if len(decoded_tokens[row]) == s_min and \
           row+1 < total_rows and \
           row+1 not in decoding_rows and \
           row+1 not in [p[0] for p in pending_starts]:
            # Generate tentative token with window size s_min
            prob, pend_token = speculative_generate(row+1, 0) ## Tentative generation of token at position (row+1, 0)
            pending_starts.append( (row+1, pend_token, prob) )

    # --- Step 4: Cleanup completed rows ---
    decoding_rows = [r for r in decoding_rows 
                        if len(decoded_tokens[row]) < total_columns]

Thanks to the reviewer for the valuable comments.

Q1: Essential reference VAR is not discussed. As noted in our related work section (lines 157-160 in the paper), we do discuss VAR and its approach to visual generation. Moreover, it should be noted that VAR requires specialized multi-scale tokenizers and must be trained from scratch as a complete generation framework. In contrast, our proposed ZipAR is a training-free, plug-and-play parallel decoding solution for existing vanilla (raster order) next-token autoregressive visual generation models without any architectural modifications or retraining.

Q2: Missing speed comparison with VAR and MaskGIT. As noted in Q1, ZipAR aims to accelerate existing vanilla next-token AR visual generation models, which is not directly comparable with VAR or MaskGIT. However, to address this concern, we have evaluated the generation efficiency of ZipAR, VAR and MaskGIT with similar model sizes. The results are presented below. Compared with vanilla next-token AR models, ZipAR greatly improves generation efficiency and narrows the efficiency gap with VAR and MaskGIT without any additional training.

Q3: More benchmarks are needed to prove the robustness of ZipAR. To address this concern, we have expanded our evaluation by assessing ZipAR’s performance using multiple metrics, including VQAScore, Human Preference Score v2, ImageReward, and Aesthetic Score, across three models: LlamaGen-XL-512, Lumina-mGPT-768, and Lumina-mGPT-1024. The results presented below demonstrate that our method significantly improves generation efficiency with little impact on output quality across various benchmarks.

Q4: Sec 3.3 can be more informative. To clearly demonstrate the details of ZipAR, we have provided a pseudo code for ZipAR's sampling process. Please refer to Q4 in our response to Reviewer CC9X due to the character limit here. We will include it in the revised version.

Thanks to the reviewer for the valuable comments.

Q1:More diverse automatic evaluation approach should be considered. To address this concern, we have expanded our evaluation by assessing ZipAR’s performance using multiple metrics, including VQAScore, Human Preference Score v2, ImageReward, and Aesthetic Score, across three models: LlamaGen-XL-512, Lumina-mGPT-768, and Lumina-mGPT-1024. The results presented below demonstrate that our method significantly improves generation efficiency with little impact on output quality across various benchmarks.

Q2: Ablation studies on whether ZipAR affects the optimal token-sampling-hyperparameters. We performed a grid search to determine the optimal token-sampling hyperparameters, namely, sampling temperature and classifier-free guidance scale, for ZipAR. The results are shown below. Here, "*" denotes the results obtained from LlamaGen's paper. These results indicate that ZipAR sampling does not alter the optimal sampling temperature and classifier-free guidance scale.