TopoDiffusionNet: A Topology-aware Diffusion Model

We thank the reviewer for the encouraging and insightful comments. Please find our responses to specific queries below.

Q1: The lower part of $\alpha_3$ seems to be cut off by a straight line?

A1: The straight line is an intermediate effect visible during the diffusion process. Specifically, in this timestep, our $\mathcal{L}_{\text{denoise}}$ loss is actively preventing the merging of $\alpha_3$ and $\alpha_2$ components by enforcing lower intensity values at the pixels between these structures. This creates the appearance of a straight 'cut-off' line below $\alpha_3$. This effect is prominent between $\alpha_3$ and $\alpha_2$ components due to their proximity and the strength of the topological constraint in this region. It is not observed below $\alpha_1$, where there is no risk of components merging. This distinction is clearer in the $\mathcal{S}(0.21)$ image, showing how the topology-enforcing mechanism maintains separation between components.

Q2: Discuss higher-dimensional topological control, such as 3D, as future work.

A2: Thank you for the suggestion. Our current framework using persistent homology can theoretically be extended to higher dimensions, as persistence diagrams can capture topological features in arbitrary dimensions. For 3D applications, we can control not just connected components (0-dim) and holes (1-dim), but also voids (2-dim). As future work, we are currently investigating 3D point clouds and volumetric medical imaging data, where maintaining topology is important in generating realistic synthetic data. We have updated the same in Appendix N (page 27).

Q3: Explain in more detail how persistent homology is calculated; the persistence diagram could be more explicit.

A3: We would be happy to expand on the persistence homology and diagram computation. As mentioned in the implementation details, we use the Cubical Ripser library [1] for the homology computation. Their manuscript has details about how they optimized the implementation. For 0-dim, tracking the birth (creation) and death (merging) of connected components is done via the union-find algorithm. For 1-dim and higher features (e.g., loops and voids), a sparse coboundary matrix is conducted that encodes relationships between pixels. This matrix is reduced to upper-triangular form using column operations, optimizing the identification of the birth and death of cycles.

In Appendix I (page 21), we now provide a detailed walkthrough example of how the 0-dim persistent diagram is computed using the union-find operations. Fig. 13 (Appendix I, page 22) consolidates the process.

Thank you very much for your review. We hope we were able to clarify your comments, and we would be happy to discuss further!

[1] Shizuo Kaji, Takeki Sudo, and Kazushi Ahara. Cubical ripser: Software for computing persistent homology of image and volume data. arXiv preprint arXiv:2005.12692, 2020.

We thank the reviewer for the encouraging and insightful comments. Please find our responses to specific queries below.

Q1: Empirically (not in theory), can your code generate more complex specific shapes like double annulus?

A1: By combining multiple topological constraints, our method can provide more sophisticated control. To show this, we conduct experiments using both 0-dim and 1-dim topological constraints simultaneously. We represent these joint constraints as pairs $(a,b)$, where $a$ specifies the number of connected components (0-dim) and $b$ specifies the number of holes (1-dim). The performance from Tab. 9 (Appendix K, page 24), or, the second table in the Global Response, shows that TDN can handle joint constraints.

A double annulus consists of concentric circles, with $(a,b) = (2,2)$. Our loss, based on homology, cannot distinguish whether two loops are positioned concentric or not, however, it can generate structures that are homotopy equivalent to the double annulus. As shown in Fig. 15 (Appendix K, page 24), by sampling for the $(2,2)$ condition, TDN can generate nested circles (visually the closest constraint to a double annulus) and other homotopy equivalent structures. This experiment demonstrates that our method can handle complex topological specifications through the combination of constraints, though geometric relationships (like equidistance and concentricity) remain an interesting direction for future work.

Q2: Topological control for practical uses, beyond the trivial case of controlling the number of objects.

A2: While counting may seem trivial for humans, it is already a challenging task for diffusion models, and in general generative models. As shown in Tab. 1 and Fig. 6, even state-of-the-art models like DALL·E 3 and Stable Diffusion fail to reliably generate a specified number of objects. Our novel approach provides control in both counting (0-dim topology), and number of holes (1-dim topology). This is a significant first step towards semantic control of generative models, moving beyond qualitative attributes such as style and texture. As future work, we are currently investigating 3D point clouds and volumetric medical imaging data, where preserving topology is important in generating realistic data.

Thank you very much for your review. We hope we were able to clarify your comments, and we would be happy to discuss further!

We thank the reviewer for the constructive feedback. The questions help us improve the justification of our work. Please find our responses to specific queries below.

Q1: The masks produced by the diffusion model in this paper lack detail. Hence, are diffusion models critical to the success of this method?

A1: While the masks generated by our method TDN may appear simplistic, they effectively capture the overall structure of the objects, and are in fact, not trivial to generate. Furthermore, the details of the TDN-generated masks are necessary and sufficient for generating realistic and semantically correct images.

Despite being binary images, these masks are difficult to generate due to variations in the number of objects, their shapes, sizes, and unconstrained spatial locations (e.g., not fixed to the center of the image). Similarly, the diverse orientation and connectivity patterns make the 1-dim datasets (road networks, neuron membranes) challenging. This is evidenced by the results below (in Q2): we show that a conditional VAE cannot learn to generate satisfying masks, even with the topology-preserving loss. Diffusion models are thus critical to the success of our method as they effectively capture spatial arrangements and object shapes (in 0-dim), and connectivity patterns (in 1-dim).

Moreover, the details in the masks generated by TDN are necessary for generating realistic images. As suggested, in Q3, we report results using basic shapes (e.g.: circles and squares) as masks to guide ControlNet. These simplified masks, although have the correct topology, result in poor-quality images. In contrast, TDN-generated masks provide much more detailed geometric, spatial and topological control which are critical to the generation of realistic images.

Finally, we stress that TDN-generated masks are sufficient to generate high quality images, thanks to the power of ControlNet. This is validated by our quantitative results (see Tab. 8 (Appendix J, page 23), or, the first table in the Global Response). We observe similar visual quality between TDN-generated masks and ground truth (GT) masks (low FID [Mask] scores), as well as similar visual quality between images generated using these masks (comparable FID [Image] score).

Overall, our masks strike a balance between being oversimplified and unnecessarily complex, and diffusion models are capable of filling in the finer textural details based on these masks.

Q2: Can VAE be just as effective at generating masks to condition ControlNet?

A2: We address this concern by conducting experiments using a Conditional VAE (CVAE) as the generative model. Fig. 16 (Appendix L, page 25) shows that the quality of the masks generated by CVAE is severely degraded, with and without our topology-preserving loss. In the table below, CVAE also has significantly worse FID compared to our TDN-generated masks, indicating a poor visual resemblance to the true masks. CVAE also struggles to preserve topological constraints, as evidenced by the low performance on metrics like Accuracy and F1.

We also attempted to add our proposed topology-preserving loss $\mathcal{L}\_{top}$ to CVAE. As shown in the table, this approach (CVAE + $\mathcal{L}\_{top}$) does slightly improve performance over the standard CVAE. However, the results remain far inferior to those of diffusion models (ADM-T and TDN). Specifically, for 0-dim datasets, it generates fragmented objects and fails to preserve overall shapes, while for 1-dim datasets, it fails to generate connected structures.

This is not surprising. CVAE’s bottleneck layer compresses information, losing critical spatial details needed to enforce strict topological constraints. In contrast, diffusion models’ design of gradual denoising steps, as well as intact image resolution, ensure the preservation of details like shape, with topology being preserved because of our novel loss.

We sincerely thank reviewer GDUJ for their time and effort in reviewing and discussing our paper. We hope our previous response has addressed all of the reviewer’s comments. As the rebuttal deadline approaches, we would be grateful to know if there are any additional questions that we can clarify; we would be happy to discuss further. Thank you.

Sincerely, Authors#3999

We thank the reviewer for the encouraging and insightful comments. Please find our responses to specific queries below.

Q1: Insight into the trade-off between control and quality. Reporting FID on all datasets.

A1: This is a valid question. Our method TDN achieves improved topological control without sacrificing the overall visual quality of the generated images. We demonstrate this by providing FID scores for all the datasets in Tab. 8 (Appendix J, page 23), or, the first table in the Global Response. Recall that there are two generation outputs: masks, and images conditioned on the masks. Our method TDN focuses on generating masks with correct topology, and delegates to ControlNet to generate images conditioned on the masks. We report both the FID of the mask (FID [Mask]) and the FID of the images generated by ControlNet when using these masks (FID [Image]).

Our analysis shows that across all datasets, the masks generated by TDN closely resemble the ground truth (GT) masks, in both visual quality (low FID [Mask]) and topology (Tab. 1). We also observe high visual quality in images generated using the masks from TDN (low FID [Image] compared to baselines).

Additionally, to evaluate the impact on final image quality, we compare the FID [Image] metric between two scenarios: ControlNet using GT masks (ControlNet + GT) versus ControlNet using TDN-generated masks. The results demonstrate comparable FID scores, indicating that using TDN-generated masks as conditioning does not degrade the image quality. In other words, TDN-generated masks are as good as GT masks for generating real images.

Q2: Would it be possible to combine 0-dim and 1-dim constraints?

A2: This is an interesting set of constraints. In our work, we focused on using either 0-dim (number of objects) or 1-dim (number of holes) constraints individually, depending on the property of the dataset. However, as suggested, we conduct experiments on synthetic data using both 0-dim and 1-dim constraints simultaneously. The constraint is of the form $(a,b)$ where $a$ denotes 0-dim while $b$ denotes 1-dim. Sample images generated by TDN are shown in Fig. 14 (Appendix K, page 24). We report quantitative results in Tab. 9 (Appendix K, page 24), or, the second table in the Global Response.

We find that TDN is capable of handling these joint constraints. While the performance is slightly lower compared to using each constraint individually, TDN still significantly outperforms ADM-T across all metrics. This shows that persistent homology (PH) can enhance the performance of the base diffusion model, even under multi-constraint scenarios. This finding opens up interesting possibilities for future work in handling even richer combinations of topological constraints.

Q3: Computational performance of the method.

A3: We use persistent homology only in the training phase to preserve the desired structures and penalize the extraneous ones. During inference, persistent homology is not used, hence there is no additional computational overhead or impact on inference time.

Q4: Is it possible to tune homology parameters to promote also bigger or smaller components?

A4: Good question. Yes, indeed, it is possible to tune parameters to control the size of the components. Given a mask, by flipping the foreground and background, applying a distance transform, and computing the 1-dim persistence diagram, the death time would reflect the size or radius of the object. This works because with the distance transform, the local maximum value corresponds to the farthest distance from the boundary, effectively encoding the object’s radius. Encoding the size constraint in this manner is interesting but could provide relatively limited flexibility. Much richer geometric prior can be injected using other topological tools such as persistent homology transform [1]; this will be an exciting direction to explore in future works.

Thank you very much for your review. We hope we were able to clarify your comments, and we would be happy to discuss further!

[1] Turner, Katharine, Sayan Mukherjee, and Doug M. Boyer. "Persistent homology transform for modeling shapes and surfaces." Information and Inference: A Journal of the IMA 3, no. 4 (2014): 310-344.