Controllable Data Generation via Iterative Data-Property Mutual Mappings

• This paper has a severe problem for deriving the training objective of $\mathcal{L}_2$. Here, $\mathcal{L}_2 = \mathbb{E}_q[-\log p(y|x)] = \int - q(z, w|x, y) \log p(y|x) dz dw = - \log p(Y=y|X=x)$, where $x, y \in \mathcal{X}_1, \mathcal{Y}_1$ are real samples but not generated one. This is because $-\log p(Y=y|X=x)$ This term is then just constant since it does not depend on any parameters $\theta, \phi, \gamma$.

• For the above reason, the proposed generative model does not maximize the variational lowerbound of the joint likelihood $p(x, y)$. This does not guarantee sample generation quality of the generative models.

W1. Increased Complexity: The proposed approach, while comprehensive, seems to add multiple components and constraints to the training process. This complexity might make it difficult for practitioners to adapt and integrate the framework into existing pipelines. Furthermore, the merger of control parameters with generative models could complicate model design, training, and deployment.

W2. Scalability Concerns: The iterative training procedure, while innovative, might raise questions about scalability and computational efficiency when applied to large-scale datasets. Some insights or experiments on how the framework scales with data size or complexity would be valuable for potential users.

W3. Diverse Datasets Testing: The method's robustness and generalizability could be further validated by testing on diverse datasets. Specifically, they could employ datasets with intricate features (like human facial datasets) to further validate the claim of disentanglement between the controlled properties and the unrelated latent variables.

W4. Limited Competitor Analysis: While the authors compare their methods with existing VAE-based generation models, it would strengthen the paper if more competitors, especially advanced disentangled VAEs were incorporated into the analysis, such as:

• Joy, T., S. Schmon, P. Torr, S. Narayanaswamy, and T. Rainforth. "Capturing label characteristics in VAEs." In ICLR, 2021.
• Ren, Yurui, Ge Li, Yuanqi Chen, Thomas H. Li, and Shan Liu. "Pirenderer: Controllable portrait image generation via semantic neural rendering." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13759-13768. 2021.
• Wang, Shiyu, Xiaojie Guo, Xuanyang Lin, Bo Pan, Yuanqi Du, Yinkai Wang, Yanfang Ye et al. "Multi-objective Deep Data Generation with Correlated Property Control." Advances in Neural Information Processing Systems 35 (2022): 28889-28901.

The significance of this work becomes apparent when considering it in the context of state-of-the-art generative models. There is a clear need for this investigation to ascertain how the proposed framework contributes to the field and whether it offers substantial improvements or unique capabilities compared to existing models.

The presentation of the work, unfortunately, poses challenges in terms of its clarity and comprehensibility. It is evident that the way the research findings and methodology are conveyed may need refinement to enhance accessibility and ease of understanding for readers.

The unnecessary introduction of mathematical operators, which may not directly enhance the quality of the contribution, can potentially obscure the core concepts and findings of the research. Simplifying and streamlining the presentation might be beneficial.

A notable concern is the utilization of extremely low-quality datasets with limited relevance. This choice could potentially hinder the model's effectiveness and real-world applicability. The inclusion of more representative and high-quality datasets may be crucial to improve the robustness and practicality of the framework.

Major Comments :

1. The authors have proposed generating the latent vector (w) associated with a given property by utilizing both the sampled latent factor (z) and the desired property (y). However, the rationale for incorporating (z) in this mapping function remains unclear, particularly given that prior works[1] in the field have only used y. It is important for the authors to provide an explanation for why both z and y are considered in the mapping function and elucidate their impact on the overall results.

2. The authors should clarify the impact of choosing different values of y on the quality of the generated output, especially when the selected values for properties (y) deviate significantly from the original dataset.

3. It seems that the authors have incorporated out-of-distribution data in a manner that resembles various data augmentation techniques, like Mixup [2], aimed at improving performance. While they have demonstrated the potential of this method for generating new molecules, conducting similar experiments on datasets such as dsprites, 3D shapes, and other real-world datasets would provide valuable insights into the impact and advantages of this strategy for generating new data, as opposed to generating intermediate data points in interpolation.

4. The utilization of various constraints in the overall training of the model lacks clarity in the paper. The authors should provide more comprehensive details about these constraints and explain how they influence the overall results.

5. The authors should provide clear notational explanations for terms like "e," particularly in the context of L4 (loss), to enhance the reader's understanding.

6. The authors have provided information on how well their proposed method preserves the desired properties in the generated output. However, there is a lack of information regarding the impact on the quality of the generated images and molecules. It would be beneficial for readers to have a quantitative evaluation of these aspects, especially after incorporating out-of-distribution data during the training process.

7. Authors should clarify the prior distribution used for 'w' and 'z' in the overall optimization.

8. There has been a lot of literature discussion around the impossibility of learning disentangled representation in an unsupervised manner. The Authors have missed a seminal work in this direction [3]. This work has to be discussed and the current paper has to be positioned correctly in that context (despite the focus of this paper is not a fully unsupervised learning).

9. There has been work on disentangled learning beyond datasets such as Dsprites. For instance, real-world datasets, that have attributs can be considered to test the proposed idea. I recommend the Authors to consider such experiments, especially given the cliam regarding the "precisely controlling the properties of generated data".

10. There is a very closely related work that claims to do a similar type of stuff detailed in [4].

References: [1] Guo, Xiaojie, Yuanqi Du, and Liang Zhao. "Property controllable variational autoencoder via invertible mutual dependence." International Conference on Learning Representations. 2020. [2] Taghanaki, Saeid Asgari, et al. "Jigsaw-vae: Towards balancing features in variational autoencoders." arXiv preprint arXiv:2005.05496 (2020). [3] Locatello F, Bauer S, Lucic M, Raetsch G, Gelly S, Schölkopf B, Bachem O. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning 2019 May 24 (pp. 4114-4124). PMLR. [4] Mondal, Arnab Kumar, Ajay Sailopal, Parag Singla, and Prathosh Ap. "SSDMM-VAE: variational multi-modal disentangled representation learning." Applied Intelligence 53, no. 7 (2023): 8467-8481.