ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation

We thank the reviewer for their thoughtful suggestions! Below, we address the questions listed under "Questions for Authors", followed by further discussion on related topics.

Q1: Why are token pairs between adjacent actions assigned lower weights?

A1: First, we'd like to clarify that token pairs between adjacent actions are not always assigned lower weights compared to token pairs within an action.

For example, consider an action A with 8 features and an adjacent action B with 3 features. Then:

• The weight for token pairs between actions A and B is: $\frac{1}{8 \times 3} = \frac{1}{24}$
• The weight for token pairs within action A is: $\frac{1}{\binom{8}{2}} = \frac{1}{28} < \frac{1}{24}$

This illustrates that in some cases, cross-action token pairs actually receive higher weights than within-action pairs.

More broadly, the weighting scheme is designed to reflect the expected probability that two tokens co-occur as neighbors in the flattened sequence, where tokens in each action (set) are randomly permuted. As a result, the final weight depends on the size of the involved feature sets. For a detailed explanation, please refer to "Section 3.2.1 - Weighted co-occurrence counting".

Q2: Performance w.r.t. the number of backbone model parameters

A2: Thank you for the suggestion. We conducted experiments to study the impact of backbone model size on performance. Specifically, we evaluated three model variants with varying parameter numbers:

We tested these variants on a small dataset (Sports) and a large dataset (CDs), with results summarized below:

These results indicate that performance is influenced by both dataset size and model size (under a fixed number of tokens). On the smaller Sports dataset, the base model performs best, while the large model shows signs of overfitting. On the larger CDs dataset, the large model achieves the best performance, suggesting the dataset is sufficiently large to benefit from increased model size.

Q3: LLM-based sequential recommendation

A3: Thank you for the valuable suggestion. While our paper primarily focuses on action tokenization methods, LLM-based sequential recommendation is indeed closely related. Below is a high-level discussion of its relevance and connection to our work:

When aligning LLMs with user preferences for sequential recommendation through instruction tuning on historical action sequences, the way these actions are tokenized plays a crucial role. We identify three main paradigms:

1. Text-based tokenization: Each action is represented as a textual string, which aligns naturally with LLMs' input modality. However, this approach leads to significantly long token sequences, resulting in both tokenization inefficiency and high inference latency.
2. Dense vector representations: Actions are represented as dense vectors, typically derived from pretrained semantic encoders or embedding tables. While this method is more efficient in terms of sequence length, it faces memory and scalability issues, especially since the number of items often exceeds the typical token vocabulary size of LLMs. Aligning LLMs with these continuous representations poses challenges in both engineering and optimization.
3. Discrete tokenization: Actions are tokenized into short sequences of discrete tokens drawn from a compact shared vocabulary (usually much smaller than that of typical LLMs). This strikes a balance between token length and memory efficiency, making it a practical solution for building LLM-based recommendation systems.

We appreciate the reviewer's input and will include a more detailed discussion of LLM-based sequential recommendation in the final version, with proper citations to relevant literature.

Q4: Cross-dataset pretraining

A4: Thank you for highlighting this direction. While the transfer learning paradigm - pretraining on a diverse collection of datasets followed by fine-tuning on new datasets or platforms - has shown promising in retrieval-based recommendation methods (e.g., VQ-Rec [Hou et al., 2023]), it remains an open challenge for generative recommendation models.

To the best of our knowledge, there is currently no existing work that successfully applies generative models with action tokenization to this pretraining - fine-tuning paradigm. Nevertheless, we agree this is an important and exciting research direction. We will clarify this point in the final version and plan to explore it further in future work.

We thank the reviewer for the valuable feedback! Below, we first provide a detailed example of learned vocabulary, followed by clarifications regarding the experiments and method design.

Q1: Example of learned vocabularies

A1: As detailed in Section E, we use vector-quantized (VQ) tokens as item features. We also add an extra token per item to avoid conflicts. Thus, each item is associated with a total of 5 features: 4 VQ tokens and one extra token. The union of these tokens forms the initial vocabulary of ActionPiece. Notably, we do not use raw item IDs or raw text tokens.

To further clarify, we provide a concrete example from the Sports dataset. The item-to-feature mapping looks like:

The ActionPiece vocabulary is constructed by iteratively merging token pairs into new tokens. Each row in the table below represents a merge rule, sorted by the order in which the rules were learned:

The final vocabulary consists of 1153 initial tokens (4×256 VQ tokens, 128 extra tokens, and 1 padding token) and merging rules.

We promise to release our code and constructed vocabularies, allowing others to reproduce and extend our work.

Q2: Experiments with vocabulary size > 40k

A2: We agree that experimenting with larger vocabulary sizes would make our study more comprehensive. We conducted experiments on the Sports dataset:

As shown, increasing the vocabulary size does not consistently improve performance, suggesting that larger vocabularies may lead to overfitting.

Q3: More ablation study with SPR

A3: To further understand SPR's impact, we introduce two additional ablation variants on the Sports dataset by applying SPR to:

1. TIGER
2. Variant (2.1), which uses only the initial tokens (without merging).

As we can see, directly applying SPR leads to degraded performance in both cases. This suggests that SPR alone is not sufficient to improve generative models, regardless of whether the tokens are ordered or unordered.

Q4: Gains of contextual tokenization seem small

A4: Note that Sports and Beauty have relatively short sequence lengths (8.32 and 8.87 actions per sequence). In contrast, CDs has longer sequences, averaging 14.58 actions per sequence. Longer sequences offer more opportunities to leverage contextual information during tokenization. As expected, the performance gap between variant (2.2) and ActionPiece is more pronounced on CDs.

Q5: Handling high cardinality features like item IDs

A5: As mentioned in A1, we add one extra token per item, which allows us to uniquely index each item. This design eliminates the need to explicitly incorporate item IDs. Likewise, for other high-cardinality features, we can adopt a joint indexing mechanism as well, representing each feature using a combination of tokens from shared vocabularies.

Q6: Order within an action vs. across actions

A6: Item features such as title or price typically do not have an inherent ordering relative to one another. In sequential recommendation, the historical actions of a user are typically ordered by timestamp to capture behavioral dynamics. Therefore, we preserve and use the temporal order of actions in the sequence. Intuitively, while features within an action are unordered, the composition of those features across time can still reflect sequential patterns.

Q7: Comparison with text tokenization methods like SentencePiece

A7: While ActionPiece can be viewed as a variant of SentencePiece that relaxes the order constraints among features within each action, this relaxation is both non-trivial and beneficial. Modeling each action as an unordered set aligns better with the inherent structure of the data and leads to improved performance. To enable effective tokenization under this setup, we introduce techniques such as weighted counting and SPR. Ablation studies show that removing any of these components results in a performance drop.

Q8: Complexity of modeling contextual information

A8: We acknowledge that ActionPiece introduces additional complexity. This mirrors the evolution seen in language modeling: when subword methods like BPE were first introduced, they were also considered more complex than word- or character-level tokenization. Yet, over time, such methods proved to be significantly more effective and have become standard in modern NLP pipelines. Similarly, we argue that context-aware tokenization is a necessary step forward for effectively modeling action sequences.

Thank you very much for your time and thoughtful feedback. We greatly appreciate your constructive and insightful suggestions. Below, we address your concerns regarding the experiments, followed by additional discussions.

Q1: Inconsistent experimental settings

A1: We included results on the CDs dataset to demonstrate performance of compared methods on a large-scale dataset. However, to the best of our knowledge, there are no publicly available results of generative recommendation methods on CDs. Therefore, we carefully followed the same experimental settings used in public benchmarks such as Sports and Beauty to ensure fair comparisons.

We would like to clarify that the "CDs" dataset used in LC-Rec [Zheng et al., 2024] is from a different version of the Amazon Reviews dataset. Specifically, our work uses the Amazon 2014 version, whereas LC-Rec uses a 2018 version.

Q2: Results on datasets beyond Amazon

A2: Thank you for the suggestion to improve the comprehensiveness of our experimental evaluation. We additionally conducted experiments on another widely used public benchmark Yelp, following the experimental setting in the LETTER [Wang et al., 2024a] paper.

These results demonstrate that ActionPiece achieves better performance than the compared baselines on Yelp as well.

Q3: Results of TIGER with smaller vocabularies

A3: In our original submission, we compared ActionPiece to both (1) TIGER with larger vocabularies and (2) SPM-SID, aiming to ensure a fair comparison in terms of vocabulary size and to demonstrate that the improvements are not solely due to having more tokens.

We appreciate the reviewer's insightful suggestion and conduct additional experiments using TIGER variants with smaller vocabulary sizes:

While one TIGER variant with smaller vocabularies can perform slightly better than the numbers reported in the original TIGER paper, ActionPiece still achieves significantly better performance than all TIGER variants.

Q4: Baselines like IDGenRec and LETTER were not compared

A4: Each generative recommendation baseline in our paper was selected to represent one different tokenization paradigm, consistent with those in Table 1. Our core contribution is to introduce context-aware tokenization as a novel and promising paradigm, rather than to claim that ActionPiece is the best action tokenization method.

That said, we agree that additional comparisons are helpful. Below are the results comparing ActionPiece with IDGenRec (after fixing the data leakage issue in https://github.com/agiresearch/IDGenRec/issues/1):

The comparison with LETTER has been included in our response to Q2 above.

Q5: Tokenize inherently ordered features

A5: Thank you for highlighting this important direction. Injecting hierarchical structure into the tokenization process is indeed a challenging task for most existing action tokenization methods, especially those that rely on quantization techniques. While text-based tokenization can naturally capture such hierarchies, it suffers from tokenization inefficiencies. We will discuss these limitations and the trade-offs in the final version of the paper.

Q6: Efficiency of set permutation regularization (SPR)

A6: In terms of training, the efficiency is comparable to existing methods. The feature permutation operations are performed on the CPU and run asynchronously alongside GPU-based model updates.

For inference, while SPR introduces additional computational overhead in terms of FLOPs, the latency remains comparable. This is because the augmented versions of each test case can be processed in parallel across multiple GPUs, resulting in inference latency that is comparable to the non-augmented single-GPU setting.

Q7: Details of the inference process

A7: Our model inference process follows TIGER. The decoder autoregressively generates token sequences for the target items. During training, we use the original item features as labels without any augmentation or token merging. At inference time, we apply beam search to generate the top-ranked token sequences. The most probable token sequences (in other words, prefixes) are retained in the current beam (with beam size detailed in Table 7), and the model continues generating tokens one at a time until the desired generation length is reached.

Q8: Formatting issues in Algorithm 2 and Figure 3

A8: Thank you for your careful and detailed review. We appreciate your effort in catching these formatting issues and will address them in the final version.

We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate your recognition of the intuition, simplicity, and effectiveness of ActionPiece tokenization technique.

Q1: On experiments with industrial-scale datasets or online A/B testing

A1: We acknowledge the reviewer's concern regarding the absence of industrial-scale experiments. Due to resource constraints, we were unable to run experiments on industrial-scale offline datasets or perform online A/B tests. However, to address the scalability concern, we included the CDs dataset in our evaluation, which contains over 1 million user-item interactions. To our knowledge, this dataset is among the largest publicly available datasets used for studying generative recommendation or action tokenization methods.

Q2: On set permutation regularization (SPR)

A2: SPR benefits the model from multiple perspectives:

• Token utilization perspective:

  SPR effectively prevents the features of a single action from being consistently merged into the most compressed (high-level) tokens. Instead, it allows the action to be tokenized into both high-level and low-level tokens, depending on the permutation and token merging rules. This increases the number of tokens actively involved during both training and inference. As shown in Figure 5 and discussed in Section 4.4.2, SPR significantly improves token utilization - from 56.89% to 95.33% by the 5th epoch - indicating that a greater proportion of tokens are trained after applying SPR.

• Data augmentation perspective:

  From the perspective of data augmentation, SPR enriches the token sequences available for model training. Without SPR, each action sequence can only be tokenized into a single, fixed token sequence. In contrast, SPR allows each action sequence to be tokenized in multiple ways (as shown in Figure 1). While these augmented sequences preserve the same semantic information, they expose the model to richer token patterns. Training on these diverse token sequences helps the model generalize better, as evidenced by the performance of variant (3.1) in Table 3.

• Ensemble perspective:

  SPR also enables inference-time augmentation. A given input action sequence can be augmented into multiple token sequences during inference. Each sequence may yield a different ranking of the next possible items. By ensembling these recommendation results, overall performance can be enhanced, as demonstrated by variant (3.2) in Table 3 and further illustrated in Figure 6.

We thank the reviewer again for their helpful comments. We will incorporate the above clarifications and discussions in the final version of the paper.