Topo-Diffusion: Topological Diffusion Model for Image and Point Cloud Generation

Thank you very much for appreciating the novelty of the Topo-Diffusion model and constructive suggestions. We have addressed your questions below and also added the respective updates into the revision.

Response to weakness:

[-] Thank you very much for your comments! We have included [1] [2] and some other existing works in our paper. We have rewritten the introduction and related work sections with a new perspective, in which we mainly focus on the underlying diffusion mechanisms.

[-] Thank you for your suggestion. We have redrawn Figures 2 and 3.

[-] We appreciate this good suggestion. We summarize that we mainly conducted experiments on structural data (point cloud, cell images, digital images, satellites imagery) to see the effects straightforwardly. We also conducted experiments on real-world human face dataset and the results showed that our proposed Topo-Diffusion is able to enhance real world data too.

Thank you very much for recognizing the novelty of the proposed ideas and the importance of the topic and for the valuable feedback and references! We have addressed your questions below and also added the respective updates into the revision.

Response to Question 1: We thank the reviewer for suggesting these papers! We agree with this comment. Here we would like to point out that our proposed Topo-Diffusion is the first approach bridging persistent homology representations of the data and diffusion generative models (i.e., incorporates these two persistence point transformation methods into the diffusion model), and improves the robustness and performance on seven real-world and synthetic datasets. Also, we would like to highlight that our Topo-Diffusion provides flexible architecture which is capable of utilizing various types of topological summaries of the input object. We have cited all the above papers and introduced their methods/applications in the `Related Work’ section.

Response to Question 2: Regarding the "parameter-free", we mean that, compared with other deep learning models, it does not have arbitrary hyperparameters that the users have to set in order to perform the analysis. Now we have removed the "parameter-free" and modified this statement as "We introduce explicit constructions of topological summaries for persistence vectorizations through the birth-lifespan transformation using Betti curve and persistence landscape. Specifically, (i) Betti curve allows us to measure various topological information with a given filtration and does not inherit any implicit biases; and (ii) the advantages of persistence landscape are invertible, stable, and nonlinear." (see Birth-lifespan transformation in the revision).

Response to Question 3: Thank you for raising this question. To better illustrate the barcodes based on a sequence of Vietoris-Rips complexes for a point cloud, we have drawn a new figure on a point cloud with 5 points (please refer to the Figure 19 in the revision). As shown in Figure 19, the top three subfigures are snapshots of the evolving complex as threshold $\epsilon$ increases. Note that the positions of the points in the figure do not exactly reflect the interpoint distances. The distances between two points in the point cloud less than or equal to $\epsilon$ are depicted in blue. The $\epsilon$ values corresponding to the two ends of the horizontal bars mark the birth and death times of topological features. We can observe that (i) when $\epsilon=0$, $\beta_0 = 0$ (i.e., $H_0$) and $\beta_1 = 0$ (i.e., $H_1$); (ii) when $\epsilon=0.4$, $\beta_0 = 3$ and $\beta_1 = 0$; and (iii) when $\epsilon=0.8$, $\beta_0 = 1$ and $\beta_1 = 3$.

Response to Question 4: We agree with this comment. Following the Reviewer’s suggestion, we have fixed this sentence, i.e., where $K$ denotes the maximum dimension of homological features of $x_t$, and updated it in the revision.

Response to Question 5: Thank you so much for notifying us this, we have corrected these issues in the revised version.

Response to weakness: Thank you very much for pointing out this concern. We have conducted additional experiments on three real datasets, which are labeled faces in the wild (LFW) dataset, the circuit reconstruction from electron microscopy images (CREMI) dataset, and NASA satellite image dataset (NASA’s Satellite Imagery) to evaluate our proposed method (which would further strengthen our work and facilitate readers' better understanding of the applications of our Topo-Diffusion). We will conduct and provide more experimental results in the final version. In the revision, we provide results on human face dataset to demonstrate the performance of our Topo-Diffusion to enhance the image quality. Focusing on structure information, we have also conducted additional experiments on cell image and satellite image datasets and shown how the structural information in synthetic image can be captured by Topo-Diffusion.

We have listed the following quantitative results to demonstrate the performance of the proposed Topo-Diffusion model. We have provided qualitative results of generated images from CREMI dataset (see Figure 19 in Appendix) and the NASA dataset (see Figure 18 in Appendix) to demonstrate the effectiveness of including persistence homology information in the Diffusion-based model. The qualitative results show that the persistence homological features enhances the generative model to coverage much richer topological information. The diffusion model can improve the diversity of synthetic data. Topo-Diffusion can improve the quality of generated images in the aspect of capturing topological/geometrical information as well as generating high-quality images.

Table 1: Performance on the CREMI dataset.

Table 2: Performance on the LFW dataset.

Table 3: Performance on the NASA’s Satellite Imagery.

We are still training TopoGAN model on LFW dataset, and NASA satellites imagery, and we will update them into the final version once completed.

Thank you very much for recognizing the novelty of the proposed ideas and the importance of the topic and for the thought provoking question on applying our model to more complex real world datasets! We have addressed your questions below and also added the respective updates into the revision.

Response to Question: Thank you so much for pointing this out. We have conducted additional experiments on three real datasets, which are labeled faces in the wild (LFW) dataset, the circuit reconstruction from electron microscopy images (CREMI) dataset, and NASA satellite image dataset (NASA’s Satellite Imagery) to evaluate our proposed method (which would further strengthen our work and facilitate readers' better understanding of the applications of our Topo-Diffusion). We will conduct and provide more experimental results in the final version. In the revision, we provide results on human face dataset to demonstrate the performance of our Topo-Diffusion to enhance the image quality. Focusing on structure information, we have also conducted additional experiments on cell image and satellite image datasets and shown how the structural information in synthetic image can be captured by Topo-Diffusion.

We have listed the following quantitative results to demonstrate the performance of the proposed Topo-Diffusion model. We have provided qualitative results of generated images from CREMI dataset (see Figure 19 in Appendix) and the NASA dataset (see Figure 18 in Appendix) to demonstrate the effectiveness of including persistence homology information in the Diffusion-based model. The qualitative results show that the persistence homological features enhances the generative model to coverage much richer topological information. The diffusion model can improve the diversity of synthetic data. Topo-Diffusion can improve the quality of generated images in the aspect of capturing topological/geometrical information as well as generating high-quality images.

Table 1: Performance on the CREMI dataset.

Table 2: Performance on the LFW dataset.

Table 3: Performance on the NASA’s Satellite Imagery.

We are still training TopoGAN model on LFW dataset, and NASA satellites imagery, and we will update them into the final version once completed.

We thank the reviewers for their constructive and thorough feedback on this Topo-Diffusion manuscript. Below we have addressed each of the comments in detail. We have also included these modifications into the revised manuscript.

Response to weakness: Thank you for the feedback. As suggested, we have revised the manuscript by including more details and refining the writing of the manuscript. Specifically, we have included more explanations and references of PH/TDA throughout the manuscript (including introduction, related work, preliminaries, and methodology sections). We also would like to highlight our new interpretation of the diffusion process in diffusion models from a perspective of structural information degrading and reconstruction and how those structural information can be represented by PH/TDA., included in the main body of the revised manuscript. In the introduction section, we have summarized the diffusion process from topological degradation and reconstruction. We explained the relationship between the forward process and topological feature degrading. We further apply multiple widely-used TDA features generation methods to analyze persistent homology information in diffusion. In the reverse process, we conduct the same TDA generation approach to reconstruct the image. We have discussed the technique in detail in the experiments section and demonstrated its effectiveness via a straightforward visualization on point clouds and images. We have added multiple quantitative results on various datasets to evaluate the performance of our Topo-Diffusion model.

Response to Question: We thank the reviewer for bringing this novel aspect to further our understanding of this work! We have transferred the objective of our Topo-Diffusion into a noise-prediction ($\epsilon$-prediction) and analyzed the Topo-Diffusion in the perspective of ELBO. In details, the objective of training our Topo-Diffusion model is optimized towards a noise prediction loss. In this regard, Topo-diffusion is parameterized as a noise-prediction ($\epsilon$-prediction) model: $s_\theta(x_t, Tp(x_t);\lambda)$, where $x_t$ is degraded data in forward process, $Tp(x_t)=[\phi(Dg^{(k)}(x_t))]^K_{k=0}$ is the topological summaries of $x_t$, $\lambda_t$ is noise scheduler. As the topological summaries are remained stable along the diffusion forward process, for our Topo-Diffusion model, the optimization objective can be simplified as $||s_\theta(x_t, Tp(x_t);\lambda)-\nabla logq(x_t|x_0)||\leq\sigma_{\lambda}^{-2}||\hat\epsilon_\theta(x_t,Tp(x_t);\lambda_t)-\epsilon||^2_2$, where $x_0$ is the original data, $\sigma_\lambda$ denotes a parameter associated with noise scheduler. The weighting function of topo-diffusion model is $w(\lambda)\leq\sigma_\lambda^{-2}$, which can be treated as a monotonically increasing function of $t$. Thus, we can qualitatively conclude that by incorporating topological summaries into diffusion mdoel, objective of our Topo-diffusion model equals to a weighted integral of Evidence Lower Bound Objective (ELBO). In addiction, Denoising Diffusion Probabilistic Model (DDPM) is a noise-prediction based model which does not consider any topological information in the degrading and reconstructing process. The objective of DDPM is $\sigma_\lambda^{-2}||\hat\epsilon_\theta(x_t;\lambda_t)-\epsilon||^2_2$, where $w_{DDPM}(\lambda)=\sigma_\lambda^{-2}$. Comparing Topo-Diffusion and DDPM, we have $w(\lambda) \leq w_{DDPM}(\lambda)$. Therefore, incorporating topological information enhances the diffusion process by tightening the weighting function. To better understand our Topo-Diffusion model, we have added a section of ``Appendix E: Understanding Topo-Diffusion with ELBO’’ to illustrate our method in the perspective of ELBO. We also have provided mathematical proof to support our demonstration. Please refer to Appendix E of our manuscript for more details.