VaporTok: RL-Driven Adaptive Video Tokenizer with Prior & Task Awareness

We appreciate the reviewer’s constructive feedback and have provided our responses below.

---

Q1: Comparison between reinforcement learning and supervised finetuning.

To address the reviewers’ concerns, we constructed an SFT dataset using the truncation positions derived from first-stage video complexity prior and directly optimized VaporTok’s taildrop probability. In order to ensure a fair comparison, we adjusted the token count for VaporTok-SFT to be comparable to that of VaporTok-GRPO. The comparative results are as follows:

It can be seen that, under the same token budget, VaporTok-SFT underperforms VaporTok-GRPO by a considerable margin. The main reason is that during SFT, the gradients from reconstruction and generation cannot be back-propagated to the taildrop probability module, so the taildrop probability is never optimized for either task and just aligns with data prior of the first stage.

By contrast, VaporTok-GRPO explicitly trades off efficiency against reconstruction & generation, allowing further token-count compression at the cost of only a slight drop in reconstruction/generation performance compared to baseline. In summary, GRPO offers two main advantages over SFT:

1. Gradient propagation for task objectives. GRPO circumvents the non-differentiability issue by incorporating reconstruction and generation signals into reward function of GRPO. SFT cannot do this, since one cannot label whether a sampled token sequence benefits generation, and the sampling operation itself blocks precise backpropagation of reconstruction gradients.
2. Parallel, stable training and rapid convergence. GRPO trains on many video clips simultaneously, inherently improving stability and speed of convergence (see Appendix B.2, Figure 3: the reward stabilizes after ∼8 K steps).

Finally, to address the reviewers’ concern about reward hacking observed in reinforcement learning (i.e., mode collapse, where different videos’ taildrop probabilities peak at the same index), we employ two countermeasures—stochastic parallel sampling and a diversity reward—and Table 3 in the main paper confirms their effectiveness via ablation.

---

Q2: The discussion about sensitivity and generalization about hyperparameters of the weight of different components in Vapor Reward

The purpose of this hyperparameter is simply to rescale the four rewards into comparable ranges so that they can be effectively traded off during the GRPO training.

For example, when applying GRPO in the LLM or VLM domains, all rewards are typically constrained to [0, 1]. Likewise, we map our model-based rewards (reconstruction and generation) to [−1, 1] and our rule-based rewards (efficiency and diversity) to [0, 10]. We chose these ranges because, without adjusting the video latent-space distribution during GRPO, the model-based rewards fluctuate much more strongly than the rule-based ones—assigning them a smaller numerical span thus stabilizes training.

After normalizing the scales, in Figure 4 of Section 4.3 and Figure 3 of Appendix B.2 we performed a simple sensitivity analysis on the two most volatile model-based rewards. We varied their weights to study the impact on final performance, testing settings from 1:1:1:1 → 1:1:1:0 → 1:1:0:1 → 1:1:0:0, and observed that both rewards are indispensable. Moreover, increasing their weights (from 1:1:1:1 → 1:1:2:2 → 1:1:3:3 → 1:1:4:4) consistently improves reconstruction and generation quality. These analyses confirm not only the effectiveness of our reward design but also that, even when an order of magnitude smaller than the rule-based rewards, the model-based terms still exert strong control over GRPO training.

As for the generalizability of this hyperparameter: different videos do not produce order-of-magnitude differences in their computed rewards. We sampled 10 % of the Kinetics-600 validation set and evaluated the rewards with our final VaporTok-GRPO; the mean values (below) show no such magnitude disparities. Thus, the same hyperparameters can be applied to different datasets without significant modification.

---

Q3: Human study to validate the effectiveness of VaporTok.

For reconstruction, we used the same 20 videos and reconstructed them with LARP (512 tokens), VaporTok (avg. 534 tokens), ElasticTok (1024 tokens), and OmniTokenizer (1280 tokens), comparing each against the ground truth. Participants saw four options and could choose two reconstrction results per video (the first choice as most preferred, the second as next preferred). Eighteen people participated in the test.

Below are the voting results, which show that VaporTok, even with a token count comparable to or significantly lower than other methods, still outperforms them.

For generation evaluation, we randomly selected 10 classes from UCF101. For each class, we generated one video with OmniTokenizer, LARP, and VaporTok. Participants then chose which model produced the best result. Eighteen people took part in the test. It can be seen that, with a comparable token count, our method achieves superior user preference.

We thank the reviewer for acknowledging the novelty of our method and the effectiveness of its efficiency. Below, we provide our responses to the reviewer’s questions.

---

Q1: What do “fixed-rate baseline” and “naive taildrop” refer to in the abstract of the paper?

The terms 'fix-rate baseline' and 'naive taildrop' mentioned in the abstract refer to the third and fourth rows in Table 2 of the main paper, respectively. Our probability taildrop experiment was configured with a maximum token length of 1024. However, since we utilize adaptive token count during both training and inference, the actual average number of tokens used is less than 1024, averaging around 500 tokens. The comparison of LARP in table1 of main paper is not a fair comparison, as LARP is using a larger number of tokens, so we have constructed a fairer comparison below (with token counts of around 500). We mitigated the shortcoming of naive taildrop causing rFVD degradation. While achieving rFVD comparable to the original baseline, our approach achieves a better gFVD and lower gFVD/rFVD ratio, demonstrating that we effectively reduced the training & inference discrepancy in the AR model through this method.

---

Q2-1: Why does naive taildrop still use 1,024 tokens during inference?

In Appendix C.4, we provide a detailed description of the training and inference procedures for naive taildrop. Specifically：

During training, naive taildrop samples a truncation index uniformly at random, discards all subsequent tokens, and feeds the preceding tokens into the decoder for reconstruction.

During inference, one may apply certain heuristics to determine how many tokens to use for reconstruction (e.g., the target reconstruction threshold employed by ElasticTok), or simply reconstruct using all tokens. Because reconstruction with the full token set always yields better performance than using only a subset, the results we report for naïve taildrop on the evaluation set are obtained by reconstructing with all 1,024 tokens.

---

Q2-2: Why does naive taildrop cause a degradation in rFVD performance?

Leveraging taildrop’s semantic-to-detail latent-space properties comes with the drawback that naive taildrop reduces rFVD. For example, the comparison between TiTok[1] and One-D-Piece[2] (TiTok trained with naive taildrop), as shown in Table 2 and Figure 4 of the One-D-Piece paper, demonstrates that using the same number of tokens leads to a drop in rFVD.

• TiTok-L with 32 tokens: rFVD increases from 2.21 (TiTok) to 3.23 (One-D-Piece)
• TiTok-B with 64 tokens: rFVD increases from 1.71 (TiTok) to 2.39 (One-D-Piece)
• TiTok-S with 128 tokens: rFVD increases from 1.70 (TiTok) to 1.96 (One-D-Piece)

As for the causes of this reconstruction performance degradation, they may include the following:

• One-D-Piece attributes this phenomenon to the fact that too few tokens cannot deliver good perceptual quality. Likewise, applying naive taildrop to compress the full set of 1,024 tokens when representing a 16-frame video may similarly undermine its representational capacity. Our probability taildrop, by contrast, compensates for the rFVD degradation caused by taildrop by adjusting the number of tokens according to the visual complexity of the data, thereby alleviating this issue.
• Moreover, current methods apply taildrop on the discrete latent representations—i.e., they perform VQ, immediately follow it with taildrop, and then reconstruct via the decoder. This incurs two successive levels of information loss (the per-token quantization loss from VQ and the sequence-level information loss from taildrop), making it more difficult to optimize the VQ codebook. We leave this aspect for future exploration.

[1] An Image is Worth 32 Tokens for Reconstruction and Generation

[2] One-d-piece: Image tokenizer meets quality-controllable compression.

---

Q3-1: Does the taildrop probability after GRPO align with that after first stage only optimized by data prior ?

During the GRPO optimization process, a KL divergence loss term is included to ensure similarity. The KL divergence between the old and new taildrop probability in GRPO loss function ensures that the multi-task-aware taildrop probability remains anchored close to data-prior taildrop probability learned in the first stage.

To verify the above statement, we compute the similarity between the taildrop probability before and after optimized by GRPO over all samples in the UCF101 validation set using the following definition (higher score indicates that the distributions are more similar)：

$

S_m &= \max\bigl(0,100 - \min\bigl(8 |m_a - m_b|,100\bigr)\bigr) ,\quad\text{where} \quad m_a= \mathrm{median}(a), m_b= \mathrm{median}(b),\

$

$

S_d &= \bigl(\min\bigl(100,100 r_d\bigr)\bigr)^{1.2} ,\quad\text{where} \quad \mathrm{IQR}_a= Q_3(a) - Q_1(a), \mathrm{IQR}_b= Q_3(b) - Q_1(b), r_d= \frac{\min(\mathrm{IQR}_a,\mathrm{IQR}_b)}{\max(\mathrm{IQR}_a,\mathrm{IQR}_b)}

$

$

S_{\mathrm{KS}} &= \max\bigl(0,100 - \min\bigl(150 D, 100\bigr)\bigr) ,\quad\text{where} \quad D = \sup_x\lvert F_a(x) - F_b(x)\rvert

$

$

S_p &= \max\bigl(0,100 - \min\bigl(30 |p_a - p_b| ,100\bigr)\bigr) ,\quad\text{where} \quad p_a= \mathrm{peak}(a), p_b= \mathrm{peak}(b).

$

$

Similarity Score &= \tfrac{1}{4}\bigl(S_{m} + S_{d} + S_{\mathrm{KS}} + S_{p}\bigr).

$

The results, shown in the table below, indicate that for the vast majority of samples the two distributions exhibit very high similarity. Samples with lower similarity reflect cases where GRPO has performed more extensive exploration in order to reconcile the four distinct reward components. Perceptually, the smooth Gaussian distribution learned in the first stage undergoes peak attenuation and a shift of its mean toward fewer tokens—thereby increasing the opportunity for more tokens to be sampled and becoming more efficient—which is consistent with GRPO’s exploration&exploitation tradeoff.

---

Q3-2: Does the taildrop probability after GRPO reflect data simplicity or complexity?

The taildrop probabilities after GRPO not only align with those learned in the first stage, but also genuinely reflect the complexity of the data. We apologize that, due to this year’s NeurIPS policy, we are unable to upload visualization figures; however, we can explain this from two perspectives:

From a quantitative perspective, the taildrop probabilities learned in the first stage fully capture the data’s complexity and the GRPO-refined taildrop probabilities are highly similar to those initial values, so the GRPO-refined probabilities likewise reflect the complexity of the video data.

From a qualitative perspective, for example, in a violin-playing video where primarily only the hands move, the mean taildrop probability is around 200 tokens; by contrast, in a soccer video with complex backgrounds and large-scale motion, the mean rises to approximately 700 tokens.

---

Q3-3: Whether cold-starting the module using the current visual prior is both expected and necessary.

As noted in our responses to Q3-1 and Q3-2, this data prior serves the role we "expected". As for "necessary", we provide the following explanation:

• Just as large language models undergo pretraining and supervised fine-tuning before GRPO to establish a strong foundation, we also require a well-initialized distribution to GRPO. This initialization stabilizes GRPO’s exploration and accelerates convergence.
• Since our reward does not include any data-specific term, GRPO alone only trades off efficiency against reconstruction & generation. Without data-prior constraint in the first stage, the final adaptive distribution would not correlate strongly with the underlying data complexity.

In summary, we want our tokenizer to be both data-prior-aware and task-metrics-aware, so we inject the data prior in the first stage. Then, in the second stage, the KL divergence term in the GRPO loss guarantees that while the distribution is adjusted toward task objectives, it retains the information encoded by the data prior. These two phases thus complement one another to achieve our overall goal.

We thank reviewer for the constructive comments. We provide our feedbacks as follows.

---

Q1: Compare with more baseline models which also use adaptive tokenizers.

We have already discussed adaptive visual tokenizer approaches such as Vilex[1], FlexTok[2], One-D-Piece[3], and ElasticTok[4] in the related work section of paper; any additional methods we have not mentioned will be included in future versions.

Vilex, Flextok, and One-D-Piece are adaptive image tokenizers and therefore cannot be directly compared to video tokenizers under a single unified metric. Moreover, ElasticTok only addresses the reconstruction task and does not include a generative component, so we compare it solely on reconstruction performance. To broaden our evaluation, we additionally include Cosmos-Tokenizer-DV[5] and a contemporaneous method, AdapTok[6]. The comparison results are shown below:

For AR-style downstream generation tasks, we additionally incorporate Video-LaVIT[7] to complement the gFVD metric. The comparison results are as follows:

[1] Visual lexicon: Rich image features in language space

[2] Flextok: Resampling images into 1d token sequences of flexible length

[3] One-d-piece: Image tokenizer meets quality-controllable compression

[4] Elastictok: Adaptive tokenization for image and video

[5] Cosmos-Tokenizer: A suite of image and video neural tokenizers

[6] Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space

[7] Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization

---

Q2-1: Analysis of the computational complexity of probabilistic taildrop during both training and inference.

During training, VaporTok learns a Taildrop Probability Query Module (22 M parameters) in addition to the original VAE (173 M parameters). This does introduce extra overhead, but only increases the total training parameters by 12.7%. The module is trained in both stage1 and stage2.

During inference:

• The Taildrop Probability Query Module runs in parallel with token discretization, adding computation but without noticeably increasing inference time.
• In the decoder, because VaporTok decodes a shorter sequence than the full length, for single-sample reconstruction one can simply truncate (rather than mask) the input before feeding it to the decoder, achieving baseline-equivalent speed.

We evaluated inference time on the UCF101 validation set with a batch size of 1, averaging over five runs. The results are as follows:

---

Q2-2: Will the introduction of adaptive encoding based on reinforcement learning make the model significantly more difficult to optimize and train to converge?

Adaptive encoding based on reinforcement learning does not make the model more difficult to optimize and train to converge. As shown in Appendix C.1, our results demonstrate that after roughly 8,000 steps, the reward values settle into a stable range, so convergence issues do not arise. We analyzed the cause and found that having a good initialization distribution before GRPO is critically important.

If one trains from scratch and simultaneously optimizes both the truncation distribution with GRPO and the latent‐space distribution, convergence can indeed be problematic.

However, VaporTok initializes with a pre-trained weights, at which point the VAE has already learned a strong latent‐space representation. On this foundation, VaporTok learns the taildrop probability from the data prior while concurrently adjusting the latent‐space representation to follow a semantic-to-detail ordering. The data prior is more stable and provides an excellent reference for GRPO; moreover, the KL-divergence term in GRPO’s objective ensures that optimization remains centered around the taildrop probability of first stage, thereby enhancing training stability.

---

Q3: More training details should to be provided.

We have already detailed the training hardware and durations in Appendix E; we now summarize them as follows:

• VaporTok-Stage1: VaporTok initializes with pre-trained weights of LARP and traines for 30 epochs on the UCF101 and K600 datasets, which requires 90 hours on 8 A100 GPUs.
• VaporTok-Stage2: The GRPO training process uses the UCF101 dataset for a single epoch, which takes 3 hours on a single A100 GPU.
• Generation: The generation task is trained on the UCF101 dataset for 3000 epochs, which takes 40 hours on 8 A100 GPUs.

The baseline model(LARP) is trained for 150 epochs on the UCF101 and K600 datasets and his generation task is also trained on the UCF101 dataset for 3000 epochs, which is as same as ours. However, LARP does not specify the GPU hardware used or the precise duration of training, so we cannot compare training durations with it.

We thank the reviewer for their recognition of our work and for the constructive comments. We provide our feedback as follows:

---

Q1-1: How does VaporTok have advantages over other methods?

Our probabilistic taildrop was configured with a maximum token length of 1024. However, since we utilize adaptive token count during both training and inference, the actual average number of tokens used is less than 1024, averaging around 500 tokens. To enable a fairer comparison, we primarily compare our approach against the 512-token scheme in this context, as follows:

Our approach offsets the rFVD degradation caused by taildrop, achieving rFVD on par with the baseline. Furthermore, our probabilistic taildrop also exhibits the semantic-to-detail property, so it delivers superior generation performance and lower gFVD/rFVD than the baseline, demonstrating that we effectively mitigate the training-inference gap of the downstream AR generation model.

---

Q1-2: Why isn't there a larger benefit from the adaptive tokenization of VaporTok?

When training a visual tokenizer, both reconstruction and generation metrics are important; however, since our goal is to solve the AR generation problem, we place greater emphasis on generation. Taildrop is inherently unfriendly to reconstruction (as shown by the TiTok vs. One-D-Piece reconstruction results below), but by ordering the latent space from semantic to detail it makes the VAE decoder more robust to later tokens, thereby boosting generation. Our method retains this strong generation performance while cleverly overcoming taildrop’s reconstruction weakness, achieving rFVD results on par with the baseline.

This degradation in rFVD performance actually occurs in all naive taildrop methods. For example, the comparison between TiTok[1] and One-D-Piece[2] (TiTok trained with naive taildrop), as shown in Table 2 and Figure 4 of the One-D-Piece paper, demonstrates that using the same number of tokens leads to a drop in rFVD, specifically:

• TiTok-L with 32 tokens: rFVD increases from 2.21 (TiTok) to 3.23 (One-D-Piece)
• TiTok-B with 64 tokens: rFVD increases from 1.71 (TiTok) to 2.39 (One-D-Piece)
• TiTok-S with 128 tokens: rFVD increases from 1.70 (TiTok) to 1.96 (One-D-Piece)

As for the causes of this reconstruction performance degradation, they may include the following:

• One-D-Piece attributes this phenomenon to the fact that too few tokens cannot deliver good perceptual quality. Likewise, applying naive taildrop to compress the full set of 1,024 tokens when representing a 16-frame video may similarly undermine its representational capacity.
• Moreover, current methods apply taildrop on the discrete latent representations, i.e., they perform VQ, immediately follow it with taildrop, and then reconstruct via the decoder. This incurs two successive levels of information loss (the per-token quantization loss from VQ and the sequence-level information loss from taildrop), making it more difficult to optimize the VQ codebook, thereby leading to decreased rFVD performance. We leave this aspect for future exploration.

Our probabilistic taildrop, by contrast, compensates for the rFVD degradation caused by taildrop by adjusting the number of tokens according to the visual complexity of the data, thereby alleviating this issue. Furthermore, our training approach arranges the video latent space in a semantic-to-detail order and strengthens the decoder’s robustness to later tokens, thereby enhancing generation performance.

[1] An Image is Worth 32 Tokens for Reconstruction and Generation

[2] One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression

---

Q2: Does VaporTok’s latent space truly exhibit the semantic-to-detail (or coarse-to-fine) property?

VaporTok supports reconstruction with varying token counts. (For details on the distinction between probabilistic taildrop during training and inference, see Appendix C.4. In brief, during training with probabilistic taildrop we could employ three different modes to get truncation index—argmax, sample, and pre-sample—whereas at inference, to eliminate randomness, we report results obtained exclusively via argmax sampling.) Also, by directly setting the truncation index, we can control how many tokens are used for reconstruction. However, if using a fixed token count for inference, VaporTok will lose the token-efficiency advantages provided by its adaptivity.

Due to this year’s NeurIPS policy, we cannot submit a PDF to display the semantic-to-detail visualization. Instead, we implicitly demonstrate this trend through reconstruction metrics at different token counts. In addition to rFVD, we report MSE, PSNR, and LPIPS of VaporTok and LARP(512) at different token counts. With an extremely small number of tokens, VaporTok significantly outperforms LARP(512), demonstrating that its head tokens capture strong semantic information and exhibits the semantic-to-detail characteristic.

The reconstruction results of V(VaporTok) and L(LARP-512)：

As for the qualitative visualizations, our final results are nearly indistinguishable from those shown in One-D-Piece (see Figure 1 of One-D-Piece paper). The critical factor in exhibiting the semantic-to-detail characteristic is using varying token lengths during training: naive taildrop chooses lengths entirely at random, whereas we sample lengths from a learned distribution. Although this sampling can somewhat attenuate the semantic-to-detail effect, we still train with different token lengths , so VaporTok naturally displays the same characteristic with One-D-Piece.