Disentangling Regional Primitives for Image Generation

1. Presentation - My main concern is the clarity of the paper, which needs significantly more work. The paper is full of details and notations and is hard to follow. The presentation of the AND interaction is not directly related to this paper (e.g., Theorem 3.1/Corollary 3.2 are not mentioned anywhere after the preliminaries) and introduces more notation that is reintroduced later anyway (e.g., defining N twice). On the other hand, some of the notations are not defined (e.g., lambda in line 294). The introduction includes a lot of related work, and in general, there are a lot of repetitions in the text (e.g., between 4 and 4.1.). Given the current state of the paper, I couldn't verify all the proofs due to the difficulty of following the text. I recommend the authors significantly revise the paper to simplify a lot of the notations and remove redundant text.

2. Usefulness - Given that the extraction procedure is computationally expensive (worked on 6x6 patches, but what about larger images), it is not clear how practical this method is for extracting the disentangled representations. An analysis of the computational cost, the number of images, and other parameters that can make this approach practical is needed.

3. Interpretability - it is not clear that the disentangled representations are more interpretable to humans in any way. Although they are "minimal" features for regional generation, it does not mean that they are more interpretable -- they still have a complex functionality. Applying the same approach for being minimal about human-interpretable concepts (e.g., disentangling the minimal features that are responsible for foreground/background generation) will provide better interpretability and controllability to the generation process.

Despite I really like the first half of the paper, I would be somehow sceptical of the overall contribution for the following reasons.

1. Limited Quantitative Evaluation: ALL evaluations are visualisations. Quantitative metrics is indeed needed, like visual quality measurement, disentanglement testing. As it stands, we’re left guessing about the true effectiveness beyond what's visually apparent. Also, the absence of comparisons with other explainable methhods weakens the case for this approach.

2. Coarse Image Division: Dividing the images into a 6x6 grid feels too crude for explainability. There’s likely a lot of information lost with such a brutish approach. Why not use finer subdivisions or superpixels that adapt to the image content? That could make the whole decomposition more meaningful.

3. Unclear Visualization: The current visualizations fall short because the gray map overlays fail to highlight background changes, making it hard to see which regions are kept. Transparent overlays, heatmaps, or color-coding could make the results much clearer.

Weaknesses:

1. To be honest, I cannot digest many parts of the article. This includes, but is not limited to, the tutorial of introduction, the method, and the experiments. I encourage the authors to rephrase some mathmatical descriptions with intuitive explanations.
2. The term $(-1)^{|S|-|T|}$ in many equations appear strange to me, e.g., Eq. 1 and Eq. 10. Why should we make the score for $T$ negative when the difference of the number of regions in $S$ and $T$ is odd? Please give more details on this -- it is important to understand its connection to the core idea of the submission.
3. I am not sure whether the extracted regional primitives are also applicable to generate other images. Otherwise, what is the purpose of extracting primitives for a specific image? The authors are encouraged to discuss the use cases of the method more clearly. Please provide specific examples.
4. It seems that the algorithm needs enumerate each subset for a given set, which may be inefficient in practice. The authors are encouraged to discuss the time efficiency and hardware requirements specifically.
5. Is the addition defined in Fig. 1(c) element-wise? What is the tensor shape of each $\Delta f$? Does it represent a patch of it has the shape of a full image feature map, i.e., $C\times H\times W$? From my understanding, $f_0$ should have the shape $C\times H\times W$ and each $\Delta f$​ should also have the same shape for element-wise summation. If so, why does the visualization only contain a patch?
6. There is almost no quantitative results. Since reconstruction is one target, at least there should be some numerical evaluation regarding this, e.g., PSNR, SSIM, LPIPS, etc.
7. There is no comparison with other methods that also focusing on interpreting GAN. Please compare to the SOTA methods.
8. Why do the authors choose layer2 of BigGAN for validation? How does this choice influence the results? How the results might change if different layers were used
9. The writing of the beginning part of Sec. 4 is largely overlapping with that of Sec. 4.1. Overall, the presentation, in terms of clarity, organization, and distinction with previous works, can be enhanced a lot.

• The author shows primarily the visual explanation of the generated image, however, it’s unclear how each “component” interacts with another component and how the components together compose the entire image in a hierarchy. It would be better if the authors can clearify what does mean by interaction?
• The authors also claim that the primitives interactions are minimized, however, Figure 4 shows that these components are heavily overlapped. Further clarification on the experiment part would be helpful.
• When explaining the feedforward process of a decision making network, it can make people understand how the network derived the conclusion. Although understanding unconditional image synthesis is interesting, it becomes much more impactful if the author could apply the proposed technique on image generation tasks that requires more precise control, such as image editing or image generation based on textual input, etc.
• The presentation of Algorithm 1 depends on the understanding of Equation 9 and 10. It would become much clearer if the author can decouple Equation 9 / 10 and provide an algorithm by simply showing the essential operational building blocks.
• A more quantitative evaluation protocol is needed to rigorously evaluate the effectiveness of the proposed method.