Concept-Centric Token Interpretation for Vector-Quantized Generative Models

Thank you for your constructive feedback. We address your concerns:

Weak baseline comparison:

Our comparison against both random selection and the frequency-based baseline (Table 1) highlights CORTEX’s effectiveness in identifying concept-relevant tokens. While the frequency-based method offers a stronger baseline than random selection, our method achieves a greater reduction in concept probability using fewer masked tokens. This suggests that CORTEX can better filter out contextual information, whereas frequency-based selection tends to pick background tokens that are not strongly tied to the concept itself.

Threshold-based approach:

Thank you for the suggestion. While threshold-based selection can serve a similar role as the top-n or top-k strategy by selecting the most important tokens based on their importance scores, it requires careful tuning for each VQGM and IEM. In contrast, our top-n and top-k strategies provide a consistent and architecture-independent evaluation across models.

Gradient-based attribution issues:

To reduce the impact of noisy gradients, we adopt the SmoothGrad [1] principle by adding noise to the input embeddings and averaging the resulting gradients, thus producing more stable token attributions. Furthermore, our sample-level explanation $\mathcal{T}^*_{\text{concept}}$ also aggregates highly activated tokens across multiple images via Equation 5, effectively mitigating the issue of gradient saturation that may occur in individual samples.

Convergence guarantees:

We acknowledge that this issue may arise with certain concepts, but we have quantitatively analyzed the performance of codebook-level explanations in Table 5 of our appendix. This analysis demonstrates that convergence to adversarial patterns is not the dominant case.

Token interdependencies:

Our codebook-level explanation can explain the interdependencies between tokens because this method optimizes tokens within a region simultaneously, obtaining the token combination that best represents this concept. We don't optimize tokens one by one but rather optimize the token selection matrix, and the optimization process considers the interdependencies between tokens.

Multi-concept tokens:

Tokens can indeed be relevant to multiple concepts. The token importance scores (TIS) are concept-specific, allowing us to map the same token to different concepts with varying importance levels. Although a single token may be associated with multiple concepts, it can represent different concepts when combined with different other tokens.

"Concept-centric" terminology:

Our method can explain abstract concepts, not just entities or objects. For example, in Section 5, we demonstrate how CORTEX explains relatively abstract concepts like "male" and "female" which go beyond simple visual objects. Furthermore, in text-to-image generative models like Dalle, our approach can explain any concept by identifying the most relevant token combinations when concepts such as "happy" or "dangerous" are used as prompts to generate images. We thank the reviewer for this thoughtful suggestion and will consider using more precise terminology in the final version of our paper.

[1] Smilkov, Daniel, et al. "Smoothgrad: removing noise by adding noise." ICML 2017.

Thank you for your detailed review. We address your concerns and questions below:

Limited evaluation setting:
Our evaluation is not limited to VQGAN or ImageNet classes. We also include a text-to-image VQGM, DALL·E in section 5.1, where concepts are defined by prompts such as “male/female/black/white doctor”. These experiments demonstrate that CORTEX can explain arbitrary, user-defined concepts beyond fixed class labels. We also include additional results on the SOTA VQGM, VAR [1]. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.

Table: Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.

Lack of comparisons:
As the first work to explain VQGMs in their token space, we have limited baseline options for comparison. However, we also established non-trivial baselines except for the random ones, including the frequency-based baseline in Table 2 and the embedding optimization baseline in Table 5. We evaluate CORTEX with different IEM architectures and find it consistently identifies concept-related tokens, demonstrating architecture-independent interpretability.

Unclear methodology aspects:
We will improve figures and notation in the revision. Specifically:

1. Less frequently activated tokens:
   Less frequently activated tokens represent elements that contribute minimally to a specific concept. These typically correspond to background information like sky or grass that lack the distinctive characteristics of the concept.

2. Token-based embeddings:
   In Figure 3, the colored grids represent the token-based embedding. Each position contains a token from the codebook. We will include the notation for E in Figure 2 in the revision.

3. Multiple embeddings per class:
   Yes, there can be multiple possible token-based embeddings for a class like "Indigo Bird." This is why we train our IEM to identify the token combination that best represents the concept.

4. Equation 3 clarification:
   Equation (3) is inspired by SmoothGrad [2], which adds noise to the input multiple times and averages the resulting gradients to reduce the model’s sensitivity to small perturbations, yielding a more stable saliency score.

5. Max in Equation 4:
   Since each channel captures different aspects of feature importance, we use the maximum across channels to identify the most discriminative feature at each position, following the max-selection approach from Section 3.1 of [3]. To validate this, we compare max and average strategies by measuring the drop in prediction probability after masking top-ranked tokens. The max operation consistently causes greater drops, indicating it better captures concept-relevant tokens.

Table: Performance comparison across average and max operation on VQGAN-generated images.

6. Letter 'k' is used multiple times:
   Thank you for your suggestion. We will improve our notations in the revision.

7. Different metrics for bias:
   Cliff's δ provides a standardized measurement that accounts for distribution differences. Other metrics like Jensen-Shannon divergence could be used, but Cliff's δ offers clear interpretability with established thresholds for effect size (small/medium/large).

8. Concept label space:
   Yes, they are the same: the concept labels we want to explain are exactly the same as the labels (or text prompts) used to condition the VQGM.

Extraction of codebook:
We will add a section in the Appendix to provide more details.

[1] Tian, Jiang, et al. "Visual autoregressive modeling: Scalable image generation via next-scale prediction." NeurIPS, 2024.
[2] Smilkov, et al. "Smoothgrad: removing noise by adding noise." ICML, 2017.
[3] Simonyan, et al. "Deep inside convolutional networks: Visualising image classification models and saliency maps." arXiv preprint.

Thank you for your positive assessment of our work. We address your questions below:

Parameters n and k:

n=20 represents the top tokens with the highest Token Importance Scores (TIS) selected from each image, while k=100 represents the most frequently occurring tokens across all sample images. Both parameters face similar trade-offs:

If the values are too small, we might miss tokens that significantly contribute to the concept or fail to capture the full diversity of concept representations.
On the other hand, if the values are too large, we risk including irrelevant tokens that dilute the concept representation or include more background information.

Figure 4 in the main paper demonstrates how performance changes as n varies from 5 to 50, showing that as the number of masked tokens increases, prediction probability continuously decreases. However, the rate of this decreasing gradually diminishes, indicating that the initial few tokens contain the core information about the concept, while including more tokens begins to incorporate less relevant information.

Latest SOTA models

Our proposed CORTEX can be applied to any type of vector-quantized generative model.
Following your suggestions, we further validate CORTEX on the latest SOTA VQGM, VAR [1] published in NeurIPS 2024. Specifically, we trained an Information Extractor Model (IEM) using token-based embeddings generated by VAR. We mask the Top-10, Top-20, and Top-30 tokens selected by CORTEX and compare the drop in prediction probability of pretrained ViT and ResNet against randomly selected tokens. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.

Table: Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.

We will also include a discussion of models such as MAGE and FSQ in the revised version.

[1] Tian, Jiang, et al. "Visual autoregressive modeling: Scalable image generation via next-scale prediction." NeurIPS, 2024.

Thank you for your thoughtful review. We address your concerns as follows.

Limited technical contribution to VQGMs:
Our method provides a quantifiable approach to detecting bias in VQGMs and pinpointing the specific tokens responsible, enabling targeted debiasing and image editing. This interpretability can be used to improve fairness, which is a key aspect of generative model performance, by facilitating both the detection of bias and the identification of its underlying sources.

Unclear benchmark evaluation:
Here, we construct a comprehensive evaluation protocol to measure whether CORTEX-selected tokens are truly relevant to the target concept. It is worth noting that no off-the-shelf benchmark exists for evaluating token-level interpretability in VQGMs. Specifically, we conduct three evaluations:

1. Sample-level evaluation (Section 4.2 Figure 4 and Table 2):

   • Dataset: VQGAN-generated ImageNet (comprehensive visual concepts coverage).
   • Evaluation Metric: Drop in pretrained classifier accuracy (ViT/ResNet) after masking identified tokens; greater drops indicate higher importance of selected tokens.
   • Results: Masking CORTEX-identified tokens causes significantly larger accuracy drops compared to randomly or frequency-based selected tokens, confirming that CORTEX effectively identifies concept-critical tokens.

2. Codebook-level evaluation (Appendix A.4 Table 5):

   • Dataset: VQGAN-generated 10 bird categories (chosen because pretrained classifiers achieve high accuracy on these categories, ensuring reliable evaluation metrics).
   • Evaluation Metric: Increase in target label probability ($\Delta P_{\text{Targ}}$) when selecting small token regions; larger $\Delta P_{\text{Targ}}$ indicates stronger association between selected tokens and target concept.
   • Results: CORTEX consistently achieves higher $\Delta P_{\text{Targ}}$ values compared to baseline methods, indicating that CORTEX selects tokens that are more aligned with the target concept.

3. DALLE-mini bias detection (Section 5.1 Table 4):

   • Dataset: Images generated by DALLE-mini using neutral prompts ("a doctor in the hospital").
   • Evaluation Metric: Frequency comparison of tokens associated with "white doctor" vs. "black doctor."
   • Results: Tokens representing white doctors occur four times more frequently than those representing black doctors, demonstrating clear bias in DALLE-mini’s generated outputs.

We will provide a more detailed description of our benchmark and evaluation in the revision.

Comparisons with other tokenizers:
Our proposed CORTEX can be applied to any vector-quantized generative model. We chose VQGAN as our backbone since it is a highly representative method in the field.

Also, following your suggestions, we further validate CORTEX on VAR [1], a SOTA VQGM. Specifically, we trained an Information Extractor Model (IEM) using token-based embeddings generated by VAR. We mask the Top-10, 20, and 30 tokens selected by CORTEX and compare the drop in prediction probability against randomly selected tokens. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.

Table: Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.

T2I model application:
We would like to clarify that our method has been applied to a T2I model, as shown in Figure 1 and Table 4 using DALL·E.
Instead of using category IDs, we directly use the text prompt (e.g., “a black doctor”) as the concept. CORTEX identifies visual tokens that are strongly associated with concept words in the prompt, allowing us to analyze token-concept relationships without requiring predefined labels.

Innovation in interpretability methods:
First, traditional methods like CAM or Grad-CAM interpret models in the pixel space, while our method operates in discrete token space. Second, traditional methods aim to explain the classifier itself, whereas we leverage the Information Bottleneck principle to train a classification model that interprets large generative models.

[1] Tian, et al. "Visual autoregressive modeling: Scalable image generation via next-scale prediction." NeurIPS, 2024.