DiffusionSat: A Generative Foundation Model for Satellite Imagery

Thank you for your feedback and recognition of DiffusionSat’s capabilities to tackle multiple relevant generative tasks for remote-sensing data. You may find responses to your questions below:

Q: Motivation for generative foundation models + “What is the purpose of conducting research on the generative foundation model?
For natural images, generative foundation models used as priors have led to major improvements in a variety of inverse problems like inpainting, colorization, deblurring [1,2,3], medical image reconstruction [4]. Just as StableDiffusion uses a different architecture and training objective to generate images compared to ViTs that are pre-trained for natural image discriminative tasks (eg: MAE [5], MoCo [6], DINO [7] etc.), we are proposing a different architecture and training objective for remote sensing generative tasks.

Remote sensing data provides unique inverse problems that are different from those in natural images. For example, super-resolving widely available multi-spectral Sentinel-2 enables object detection and classification for regions of the world which lack high-resolution satellite data [8]. High-resolution satellite images are also crucial for crop-phenotyping [9], for conservation efforts in biology [10, 14], and landslide hazard assessment [11], 3D city modelling [12], archaeological prospection [13]. Other inverse problems such as cloud/shadow removal have important uses in urban planning [15,16,17]. Further, temporal generation and multi-spectral imputation have benefits towards precise agricultural applications [18].

These studies demonstrate the real-world importance of solving generative tasks using remote-sensing data for industrial and academic use-cases. However, the challenge is that these papers introduce solutions tailored to solve their specific task, and these solutions cannot generalize to multiple downstream generative tasks in one unified framework. We therefore require more research from the generative perspective for remote sensing and our work is a significant step towards this goal.

Q: The proposed foundation model for satellite images is not novel and significant since there are many foundation models in the remote sensing area
We believe the reviewer has misunderstood the context of our work and has made a hasty judgement brushing off DiffusionSat’s novelty by dismissing the importance of generative models. Neither of the methods the reviewer cited [19, 20] are generative, which further demonstrates the need for such large-scale generative models in this field and novelty of our work.

Claiming that our model is “not novel since there are many foundation models in the remote sensing area” is a rather simplistic view that groups all large-scale models as “foundation models” without noting fundamentally different and important distinguishing factors, i.e. generative model vs discriminative models. This is unfair to the author, other reviewers, and ACs who will take these hasty judgements into account.

To go into more detail, [19,20] are both large-scale, pretrained vision transformers which can solve downstream discriminative tasks, just like [21] which we mention in our paper as a discriminative foundation model. To ask why they cannot solve generative tasks is akin to asking why a self-supervised method for discriminative tasks such as MoCo [6] cannot generate images from captions like StableDiffusion can. They have fundamentally different theory and design/architecture.

Q: “Please explain why the existing foundation models are not applicable to the generative tasks.”
As mentioned above, the reviewer seems to conflate generative and discriminative models. Discriminative models are not designed to and thus cannot solve any of the inverse problems we mention, including super-resolution, cloud/shadow removal, temporal interpolation.

Dear reviewer, As the discussion deadline approaches, we hope that your concerns are resolved and that you can reconsider our scores. Thank you very much for your time

Q: “The methodology of training the proposed foundation model is not novel and the authors only apply the stable diffusion model to the satellite images”
We want to stress that our work is more than simply “applying stable diffusion to satellite images”. We demonstrate the effectiveness of simply fine-tuning and applying vanilla stable diffusion to satellite images in all of our experiments: in tables 1, 2, 3, 4. All those results demonstrate that StableDiffusion by itself is insufficient to successfully perform generative tasks with satellite data. For single-image generation, SD is unable to capture variation in season, location, or resolution. For super-resolution, it often “hallucinates” results. Similarly, its performance is worse than DiffusionSat for both temporal generation and texas-housing super-resolution.

We clearly demonstrate that applying stable diffusion to satellite images is non-trivial. Given the differences in satellite images with natural data that we mention in the paper (i.e. irregular time intervals, resolutions, multi-spectral bands, correspondence with “global metadata” etc.), we demonstrate the need for a more thoughtful approach towards designing a generative foundation model for satellite images.

DiffusionSat does this; our novelty is in recognizing the important ingredients towards designing the first diffusion-based generative foundation model for remote sensing data. We demonstrate the importance of metadata and associating images with numerical metadata in addition, and even in the absence of, semantic text captions. We design a very general 3D conditioning mechanism that enables DiffusionSat to generate images conditioned on a variety of control signals, including images at different time-steps, resolutions, and multi-spectral bands. Further, we demonstrate, qualitatively and quantitatively, the improvements of our method on a collection of generative tasks. The reviewer’s comments about our methodology “only applying stable diffusion to satellite images” or that it is simply “a combination of StableDiffusion and contolnet” come across as disingenuous.

We hope that our work’s novel approach spurs interest and further activity in an underexplored, but highly important field: solving generative tasks for remote sensing data via foundational models.

[1] Whang, Jay, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, and Peyman Milanfar. "Deblurring via stochastic refinement." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16293-16303. 2022.

[2] Luo, Ziwei, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B. Schön. "Refusion: Enabling large-size realistic image restoration with latent-space diffusion models." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1680-1691. 2023.

[3] Kawar, Bahjat, Michael Elad, Stefano Ermon, and Jiaming Song. "Denoising diffusion restoration models." Advances in Neural Information Processing Systems 35 (2022): 23593-23606.

[4] Pinaya, Walter HL, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F. Da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, and M. Jorge Cardoso. "Brain imaging generation with latent diffusion models." In MICCAI Workshop on Deep Generative Models, pp. 117-126. Cham: Springer Nature Switzerland, 2022.

[5] He, Kaiming, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. "Masked autoencoders are scalable vision learners." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009. 2022.

[6] He, Kaiming, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. "Momentum contrast for unsupervised visual representation learning." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9738. 2020.

[7] Caron, Mathilde, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. "Emerging properties in self-supervised vision transformers." In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660. 2021.

[8] Shermeyer, Jacob, and Adam Van Etten. "The effects of super-resolution on object detection performance in satellite imagery." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0. 2019.

[9] Zhang, Chongyuan, Afef Marzougui, and Sindhuja Sankaran. "High-resolution satellite imagery applications in crop phenotyping: an overview." Computers and Electronics in Agriculture 175 (2020): 105584.

[10] Boyle, Sarah A., Christina M. Kennedy, Julio Torres, Karen Colman, Pastor E. Pérez-Estigarribia, and Noé U. de la Sancha. "High-resolution satellite imagery is an important yet underutilized resource in conservation biology." PLoS One 9, no. 1 (2014): e86908.

[11] Nichol, Janet E., Ahmed Shaker, and Man-Sing Wong. "Application of high-resolution stereo satellite images to detailed landslide hazard assessment." Geomorphology 76, no. 1-2 (2006): 68-75.

[12] Li, Xinghua, Zhiwei Li, Ruitao Feng, Shuang Luo, Chi Zhang, Menghui Jiang, and Huanfeng Shen. "Generating high-quality and high-resolution seamless satellite imagery for large-scale urban regions." Remote Sensing 12, no. 1 (2019): 81.

[13] Beck, Anthony, Graham Philip, Maamoun Abdulkarim, and Daniel Donoghue. "Evaluation of Corona and Ikonos high resolution satellite imagery for archaeological prospection in western Syria." antiquity 81, no. 311 (2007): 161-175.

[14] Johansen, Kasper, Nicholas C. Coops, Sarah E. Gergel, and Yulia Stange. "Application of high spatial resolution satellite imagery for riparian and forest ecosystem classification." Remote sensing of Environment 110, no. 1 (2007): 29-44.

[15] Li, Xinghua, Zhiwei Li, Ruitao Feng, Shuang Luo, Chi Zhang, Menghui Jiang, and Huanfeng Shen. "Generating high-quality and high-resolution seamless satellite imagery for large-scale urban regions." Remote Sensing 12, no. 1 (2019): 81.

[16] Xiao, Jieying, Yanjun Shen, Jingfeng Ge, Ryutaro Tateishi, Changyuan Tang, Yanqing Liang, and Zhiying Huang. "Evaluating urban expansion and land use change in Shijiazhuang, China, by using GIS and remote sensing." Landscape and urban planning 75, no. 1-2 (2006): 69-80.

[17] Piyoosh, Atul Kant, and Sanjay Kumar Ghosh. "Semi-automatic mapping of anthropogenic impervious surfaces in an urban/suburban area using Landsat 8 satellite data." GIScience & Remote Sensing 54, no. 4 (2017): 471-494.

[18] Gevaert, Caroline M., Juha Suomalainen, Jing Tang, and Lammert Kooistra. "Generation of spectral–temporal response surfaces by combining multispectral satellite and hyperspectral UAV imagery for precision agriculture applications." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8, no. 6 (2015): 3140-3146.

[19] Cha, Keumgang, Junghoon Seo, and Taekyung Lee. "A billion-scale foundation model for remote sensing images." arXiv preprint arXiv:2304.05215 (2023).

[20] Wang, Di, Qiming Zhang, Yufei Xu, Jing Zhang, Bo Du, Dacheng Tao, and Liangpei Zhang. "Advancing plain vision transformer toward remote sensing foundation model." IEEE Transactions on Geoscience and Remote Sensing 61 (2022): 1-15.

Thank you for your insightful feedback, recognition of DiffusionSat’s novelty and its evaluation on multiple challenging remote-sensing generative tasks. Your suggestions to improve our presentation have been very helpful to make our contributions clearer and our paper more cogent. You may find responses to your questions below:

Q: Domain specificity of DiffusionSat, exploring its generality beyond remote sensing?
Thank you for your suggestion. Our goal is to demonstrate the importance of DiffusionSat as a foundation model tailored to tackle the unique generative tasks provided by remote-sensing datasets. For example, performing super-resolution on widely available multi-spectral Sentinel-2 data can enable object detection and classification for regions of the world which lack high-resolution satellite data [1]. Other studies demonstrate the benefits of high-resolution satellite data for crop-phenotyping [2], for conservation efforts in biology [3], and landslide hazard assessment [4]. Further studies have shown the importance of cloud or shadow removal in satellite images of urban areas [5] towards urban planning.

These references demonstrate the slew of important inverse-problems associated with remote-sensing data which have broad societal implications. They also demonstrate the growing field of using machine-learning techniques for remote-sensing data. However, each of the works above tailors a specific solution to solve the generative task at hand, and therefore that solution often only applies for their use-case. The important advantage and contribution through DiffusionSat is its potential to tackle multiple of these inverse problems at once. We provide researchers and practitioners an easy way to use DiffusionSat to solve a generative task involving remote-sensing data.

Thus, we intentionally evaluate and demonstrate the usefulness of our solution on remote sensing data, with the aim of providing a general-purpose solution and with the hope to spur future interest and research in the field of generative foundation models for this data modality.

We agree that extending and applying DiffusionSat on other forms of multispectral data would be a valuable future line of research inquiry. While we aren’t aware of any such large-scale dataset, we would be happy to accept pointers for the same for future work.

Q: Clarification on the number/types of downstream tasks
Thank you for your observation. We apologise for the structure. We want to clarify that our downstream tasks are: (1) multi-spectral input super-resolution, (2) temporal generation/prediction (3) temporal in-painting for natural disaster datasets of satellite imagery.

The single-image generation tasks (which includes generating single-images given a text prompt and metadata information) are for pre-training DiffusionSat and are not downstream tasks. We first train DiffusionSat on single-image generation, and then use this model for downstream conditional generation on the 3 tasks mentioned above.

Q: Incomplete in-painting section
Thank you for the observation. We apologise; we meant to include a description of this section that was unintentionally omitted in the first submitted version. We have included a description of the task along with the generated samples with a clearer description to aid readability. We describe our changes in the overall response. We hope this adds further clarity.

Q: Other baselines for superresolution and conditional image generation tasks
Thank you for the suggestion. Table 1 is intended to demonstrate the effectiveness of our design choices for pre-training the single-image version of DiffusionSat. We compare with StableDiffusion as the largest, most effective single text-to-image publicly available as of the time of writing (which beats most other generative baselines)[6]. Our results in-turn demonstrate the effectiveness of conditioning on text and metadata for DiffusionSat. The methods in table 2 are task specific and unlike DiffusionSat, they cannot be applied for other relevant generative tasks. For example, STSR or Pix2Pix cannot perform single-image generation given text or metadata.

We have updated Table 3 (now table 2) to include further baselines for super-resolution, based on related work. As can be seen, DiffusionSat outperforms these prior works on the super-resolution tasks. We want to emphasize that the quantitative metrics don’t reveal the full-story- metrics such as SSIM/PSNR/MSE can be high for baselines that produce very fuzzy or unrealistic samples (eg: DBPN [7]). LIPIPS better reflects visual “realism” and our qualitative results also demonstrate the high quality of our generated samples. The drawbacks of SSIM/PSNR are a known problem [8,9].

Q: “why are results are only compared against non-adapted stable diffusion”
As mentioned above, we do include additional baselines for comparison. We wanted to emphasize the strength of the StableDiffusion baseline, which has been shown across a variety of natural image generative tasks to outperform other baselines[6], including the ones cited in other tables in our work. Therefore, being able to outperform StableDiffusion by strong margins underscores the benefit of our work: our pre-training on text-metadata specific to satellite imagery unlocks generative models’ potential on the unique properties of satellite images.

[1] Shermeyer, Jacob, and Adam Van Etten. "The effects of super-resolution on object detection performance in satellite imagery." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0. 2019.

[2] Zhang, Chongyuan, Afef Marzougui, and Sindhuja Sankaran. "High-resolution satellite imagery applications in crop phenotyping: an overview." Computers and Electronics in Agriculture 175 (2020): 105584.

[3] Boyle, Sarah A., Christina M. Kennedy, Julio Torres, Karen Colman, Pastor E. Pérez-Estigarribia, and Noé U. de la Sancha. "High-resolution satellite imagery is an important yet underutilized resource in conservation biology." PLoS One 9, no. 1 (2014): e86908.

[4] Nichol, Janet E., Ahmed Shaker, and Man-Sing Wong. "Application of high-resolution stereo satellite images to detailed landslide hazard assessment." Geomorphology 76, no. 1-2 (2006): 68-75.

[5] Li, Xinghua, Zhiwei Li, Ruitao Feng, Shuang Luo, Chi Zhang, Menghui Jiang, and Huanfeng Shen. "Generating high-quality and high-resolution seamless satellite imagery for large-scale urban regions." Remote Sensing 12, no. 1 (2019): 81.

[6] Rombach, Robin, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. "High-resolution image synthesis with latent diffusion models." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695. 2022.

[7] Haris, Muhammad, Gregory Shakhnarovich, and Norimichi Ukita. "Deep back-projection networks for super-resolution." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1664-1673. 2018.

[8] Zhang, Richard, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. "The unreasonable effectiveness of deep features as a perceptual metric." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595. 2018.

[9] Saharia, Chitwan, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. "Image super-resolution via iterative refinement." IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 4 (2022): 4713-4726.

I'd like to thank the authors for their rebuttal. I think the rebuttal provides clarification and definitely improves my understanding of the paper.

I would like to highlight that I was initially sceptical about the generative downstream tasks focus of the paper, since most of the work that I know in this area focuses on discriminative downstream tasks. However, this work lists indeed relevant generative downstream tasks and might serve as a baseline for future work in this area.

In my opinion the contribution is interesting, novel, and relevant not only to the EO community. I am happy to increase the score and would like to tell the authors to continue working on this interesting topic.

Q: Varying image resolution in multi-spectral images
Thank you for your suggestion. It is true that sentinel images contain bands of varying resolution. For our super-resolution task, we consider upsampling a sentinel image with bands of GSD 10m, 20m, and 60m to a resolution of around 0.5m-1.5m. We focus on this task as one of high-importance, given the abundance of unlabelled sentinel-2 imagery around the world but the dearth of high-resolution images for multiple locations and time-stamps. Multi-spectral image generation was therefore not one of the tasks we chose to demonstrate, although DiffusionSat can handle this task as well.

We want to emphasize the generality of our approach: using our 3D conditioning scheme, we can extend DiffusionSat to a variety of control signals (eg: images, multi-spectral inputs, numerical metadata etc.). This would also include multi-spectral inputs, either for conditional generation, or as its own pretraining dataset for a multi-spectral generative model.

We also would like to mention that our approach would be able to handle multi-spectral inpainting. With a suitable dataset, we can condition on input multi-spectral bands as control signals and super-resolve them to high-resolution. However, we aren’t aware of existing large-scale publicly available datasets which would provide ground truth high resolution spectral bands for multiple different wavelengths. OUr method would be able to handle this task for existing spectral bands in our dataset (eg: RGB). That is, masking parts of one or more input bands, providing the image as conditioning input, and then reconstructing the bands is almost identical to the in-painting tasks we demonstrate.

For varying image resolution in general: we do demonstrate our method’s ability to handle such inputs. For temporal generation, images in a sequence of fMoW images are not necessarily of the same GSD. We input the GSD as metadata precisely so that the model can understand the correlation between GSD and the size of objects in the image. For the texas-housing task, we input one high resolution image (GSD 1m), and two low resolution images (GSD 10m) to form the control sequence s. DiffusionSat easily handles conditional generation given this input as it understands the different input metadata per image.

Q: Multi-spectral super-resolution task definition
Thank you for the observation. We should have been clearer. To clarify, the super-resolution task in the paper is one of super-resolving multi-spectral input images to their high-resolution RGB counterparts. With a suitable dataset, DiffusionSat would also be able to generate super-resolved spectral bands given low-resolution conditioning signals, which we leave for future work.

Thank you for your thoughtful feedback, emphasis of the significance of our problem statement, recognition of DiffusionSat’s advantages over prior works and its novelty in tackling temporal satellite-image generation. We also value your comments on improving the evaluation of our method on different benchmarks and tasks and on clarifying our presentation. You may find responses to your questions below.

Q: Technical Novelty
We would like to stress that part of the novelty lies in identifying the usefulness of metadata towards controlling useful generations from latent diffusion models. Natural images simply use semantic text captions to generate images since these are readily available. We recognize the importance of remote-sensing specific numerical metadata towards generating satellite images that are aligned with season, weather, location, resolution etc. This positively affects not only generation quality, but also allows users to prompt the model in more flexible and fine-grained manner (i.e. with/without detailed text, specific months, resolutions, etc.)

While the temporal transformer is introduced in VideoLDMs, our method uses them as part of a conditioning model. Our temporal controlnet is slightly different from the VideoLDM temporal layers; the 3D convolutions of our controlnet operate by default(i.e. they don’t have the learned mix-in parameter), but the temporal attention layers are mixed-in with a learned parameter. The reason is that we swap out 2D convolutions from the controlnet, and so we require some temporal component to always be active (that is, the 3D convolutions). We did experiment with using the temporal transformer by default, but this didn’t work as well.

We would also argue that using a 3D controlnet in this way is novel, as it unlocks the capabilities of a single-image pretrained LDM (eg: DiffusionSat) to generate arbitrarily long sequences of images auto-regressively. We can do this by either conditioning on a seed sequence of images (eg: real satellite images of a location) and then generate frames in the past, future, or in-between, or generate a completely synthetic sequence of images by sampling the first image of the sequence from single-image DiffusionSat and then using the synthetic image as the seed. In fact, we demonstrate this capability in the appendix, in figure 8, in the revision.

The combined single-image, 3D controlnet approach using numerical metadata allows DiffusionSat to be very general. Control signals of almost any format can be input to our model, whether by numerical metadata or image sequences, to generate a desired output satellite image. This approach to generative modeling for remote sensing data is novel and very flexible.

Q: Comparison to other baselines
Thank you for the suggestion. We have updated our results tables with results from other baseline models. Our results for temporal generation and super-resolution demonstrate DiffusionSat’s improvement over baselines, both in qualitative and quantitative results.

Q: Quantitative results performance
Thank you for your observation. We would like to emphasise that some of the quantitative metrics don’t reveal the full story. Metrics such as SSIM/MSE/PSNR can be optimized by samples that aren’t perceptually realistic [1,2], which can indeed be seen by our generated samples we have included in figures 4 and 5. DBPN doesn’t produce many sharp details or any realistic structures, but is able to achieve a high PSNR/MSE. Our samples demonstrate DiffusionSat’s strong ability to super-resolve on the order of 10-20x and predict the past and future changes to images, including on buildings/construction, crop maturity, season, and location. Given these advancements, we claim that DiffusionSat is state-of-the-art in many generative tasks for remote-sensing.

Q: Inpainting figure visibility
Thank you for the suggestion. We have included a new figure with more detailed generated samples for the in-painting section. Our qualitative results for in-painting demonstrate DiffusionSat’s strong capability to in-paint damaged regions across a variety of disasters. We also demonstrate the ability to introduce damage to images, based on the type of natural disaster. We hope that this tool is useful to researchers and first-responders who wish to understand damaged or obscured structures in a region in the event of a disaster (eg: which roads are flooded/blocked, which houses were razed, how the landscape changes due to disaster etc.)

Q: Formatting
We have included these changes, thank you for catching them!

Dear reviewer,

Thank you for your response, we appreciate your thoughtful feedback and discussion. We hope that your further concerns are resolved and can reconsider our scores.

Thanks!

Thank you for your constructive feedback, recognition of the novelty of our approach and the significance of our problem statement, and the wider relevance of our benchmark evaluations and datasets. We also appreciate your feedback to improve the presentation and clarity of our experimental section, and to compare our work against other relevant baselines. You may find responses to your questions below:

Q: Missing results
Thank you for your observation. We have added results comparing DiffusionSat with other baselines to the super-resolution and temporal generation tasks. We demonstrate an improvement over prior state-of-the-art super-resolution models including StableDiffusion [1], DBPN [2], and Pix2Pix [3]. We also include qualitative samples for the super-resolution task. These results demonstrate DiffusionSat’s strong capability to super-resolve very low resolution input images to reveal broad structures and landscape. Other methods typically struggle with very low resolution inputs, as can be seen with DBPN producing blurry images. StableDiffusion produces realistic-looking images, but more frequently hallucinates details.

For temporal generation, we include Masked Conditional Video Generation (MCVD) a state-of-the-art conditional video generation diffusion model. Our results demonstrate an improvement over this method, as well as over StableDiffusion adapted with out 3D controlnet. Qualitative samples reflect our method’s improvement over prior works, with more realistic images generated both in the past and the future.

We hope that these results further strengthen our case for DiffusionSat and its role as a generative foundation model for remote sensing data.

Thank you also for suggesting other super-resolution baselines. We have cited these works as part of our related work section. However, were unable to locate public code repositories in order to reproduce their method for our paper. We are happy to include the method’s results given pointers to their implementation. We have included other methods with publicly available code to ensure fair comparison with our method and avoid potential bugs from re-implementing previous works from scratch.

Q: Writing cohesion
Thank you for the suggestion. We hope to ease the reader into relevant diffusion-model terminology via the background section, which is relevant as we describe our method in the Method sections. We provide a summary of related work towards the end of the paper to aid the reader between the methods and experiments section for better cohesion.

We have updated our method section to move the description of the datasets used for control signal conditional generation to the experiments section. We have also added further detail in the appendix, including a preliminary investigation into sample quality bias across geography and image-unconditional sequence generation. We hope that these additions improve cohesion.

Q: Other baseline comparison results and more examples of generated images
Thank you for the suggestion. We have also included more baseline comparison results as mentioned above and in the overall response. In addition, we have provided samples for the temporal generation and super-resolution tasks in the paper, comparing the quality of our results with prior works.

In the appendix, we also include a brief section on geographical bias, and samples from our model for unconditional temporal generation (i.e. without conditioning on an input image). We hope that these samples demonstrate the quality of DiffusionSat.

Q: Inpainting damage figure limited visibility
We have updated this section to show more samples for the in-painting task. Our in-painting section now properly describes our qualitative results on this task. Since the task is very similar to temporal prediction, we defer the reader to refer to those quantitative results for comparison with prior works.

Our qualitative results for in-painting demonstrate DiffusionSat’s strong capability to in-paint damaged regions across a variety of disasters. We also demonstrate the ability to introduce damage to images, based on the type of natural disaster. We hope that this tool is useful to researchers and first-responders who wish to understand damaged or obscured structures in a region in the event of a disaster (eg: which roads are flooded/blocked, which houses were razed, how the landscape changes due to disaster etc.)

Q: Typos and formatting
Thanks for the suggestions, we have taken a closer look and fixed some formatting and typos.

[1] Rombach, Robin, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. "High-resolution image synthesis with latent diffusion models." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695. 2022.

[2] Haris, Muhammad, Gregory Shakhnarovich, and Norimichi Ukita. "Deep back-projection networks for super-resolution." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1664-1673. 2018.

[3] Isola, Phillip, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. "Image-to-image translation with conditional adversarial networks." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134. 2017.

Dear reviewer, As the discussion deadline approaches, we hope that your concerns are resolved and that you can reconsider our scores. Thank you very much for your time

Reference: [1] Haris, Muhammad, Gregory Shakhnarovich, and Norimichi Ukita. "Deep back-projection networks for super-resolution." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1664-1673. 2018.

[2] Voleti, Vikram, Alexia Jolicoeur-Martineau, and Chris Pal. "MCVD-masked conditional video diffusion for prediction, generation, and interpolation." Advances in Neural Information Processing Systems 35 (2022): 23371-23385.