Monsters in the Dark: Sanitizing Hidden Threats with Diffusion Models

We would first like to thank you for your response and time in sharing the above resources. We have addressed as many of your concerns as possible. Please see our responses below. (R) refers to you, the reviewer, and (A) indicates our response.

(R) "The application scenario in this article is not clearly described. The two use cases mentioned in the penultimate paragraph of the manuscript have been replaced by robust steganography, a more suitable tool. Robust steganography has been developed for many years [1~5] and is not mentioned in this article, which is a lack of research."

(A) We thank you for your feedback and appreciate you mentioning robust steganography. Robust steganography and corresponding techniques were not included in this work, as we assume in the paper that all steganographic techniques are meant to survive modifications to the container. Additionally, most robust steganographic techniques lie in the domain of traditional steganography and are used to embed text secrets or grayscale images, where we consider RGB images. We can therefore hide an image secret which is 3x larger (RGB: 3xhxw compared to Grayscale: 1xhxw).

(R) "This manuscript lacks research on relevant literature on sanitization methods for robust information hiding [6~10]. The authors should not define their previous work [11] as the state of the art easily."

(A) The authors would first like to thank you for your added literature. We have updated our literature review in the paper to include these works. In [6], the authors create a watermarked image in the training process to learn a successful attack. [7] uses STDM and SS watermarked images from BOSS to train the Fully Convolutional Neural Network Denoising Attack (FCNNDA). This approach uses text-based secrets and is also non-blind. The method proposeds in [8-mentioned in paper] and [9] use containers created with DMAS (text secrets) in the training process. This means that they are not blind and are using a different type of secret. While [10] could possible be considered blind (wasn't able to find code and steg was used to train parameters), they only consider text secrets, which is also not directly applicable to our work. These are great additions to the literature review and help to better position our paper. Thank you for linking them in your review!

(R) "At the end of the abstract, the authors propose DM-SUDS is the first blind deep learning image sanitization framework to meet these image quality results. I suggest the authors read the literature [10]."

(A) Thank you for pointing this out. We are still unsure if [10] can be considered blind as the image quality of the approach is heavily dependent on the parameter tuning, which utilizes steganographic images. We do, however, cede that we can be more precise in our language and have updated this sentence to be: DM-SUDS is the first blind deep learning image secret sanitization framework to meet these image quality results. Thank you for highlighting this!

(R) In subsection of 2.2 SANITIZATION, it is not rigorous to indicate that text-based secrets are more fragile than image-based secrets. The authors should conduct experiments to verify the robustness of text-based secrets embedded by robust steganography (such as DMAS [1]) and image-based secrets hidden by UDH [12].

(A) Thank you for pointing this out. As we focus only on image-based secrets, we have removed this blanket statement from section 2.2. We do, however, believe that image-based secrets are more robust but agree that this statement should not be included without the experimentation to back it up. Thank you! We believe this will be an interesting area to explore in future work.

(R) "In section of ANALYSIS AND DISCUSSION, the DM-SUDS also be evaluated on ImageNet. Conducting more experiments with more complex images is indeed necessary, however, the evaluation is limited only to LSB. We all know that LSB is not robust and can be disabled by common JPEG compression. It is meaningless to sanitize the secret messages embedded by LSB. LSB method can not transmit messages over lossy channels such as online social networks."

(A) Thank you for this suggestion. We have updated these experiments in the paper to be conducted on DDH and believe this greatly strengthens our analysis. Please see section 5, Evaluation on ImageNet. We would also like to highlight that not all images are or can utilize JPEG compression. There are cases where lossless image types are required, such as medical imaging and forensic imaging for legal processes. Please see our reviewer response to Reviewer zZQW.

Thank you for your review. We have addressed as many of your concerns as possible. Please see our responses below. (R) refers to you, the reviewer, and (A) indicates our response.

(R) It would good if the authors can delve into the mechanism of why diffusion model can help improve the image quality preservation of SUDS. For example, provide more discussions

(A) Thank you for your comment. We attempt to address this in Section 5, Direct Denoising. We find that the true power of the sanitization process occurs in the forward denoising process. We believe that the image quality preservation of DM-SUDS beats SUDS because of the choice of network: U-Net compared to a VAE. Where information is lost in a VAE architecture, the skip connections of a U-Net allow more relevant information to persist after the bottleneck.

(R) The methodology of this work is somehow incremental. If the authors could address more clearly about the advantages of introducing the diffusion (even with some demonstrative experiments), it would be better.

(A) Thank you for this suggestion. We believe the benefits of using DM-SUDS really shine in Section 4.2. Here, we see that DM-SUDS is better able to preserve image quality while maintaining sanitization performance.

Thank you for your review. We have addressed as many of your concerns as possible. Please see our responses below. (R) refers to you, the reviewer, and (A) indicates our response.

(R) "In my subjective opinion, authors are solving wrong problem...This means that rational attacker has a little incentive to use suboptimal methods, where better ones exists."

(A) The authors would first like to thank you for your honest feedback. In regard to steganography being considered `very detectable', we cannot argue against the performance of detection capabilities. We would like to highlight, though, that these detection models are trained on known steganographic methods. The training sets or statistical anomalies are collected from steganography techniques that already exist. Detection capabilities might then perform well at detecting containers made with these known techniques, but these help to safeguard the past. Steganography is a volatile field, especially since the application of deep neural networks to hide information introduced in [1]. This is definitely not an extensive list, but other works shortly after include [2-8]. As new steganographic techniques are being constantly introduced, how do we then protect the future as well? We believe that sanitization provides this assurance, as shown in the paper. DM-SUDS is able to protect against unknown steganography techniques (not used for training) while maintaining a high image quality. Please see Section 4.2 for this analysis. This is a very exciting result as it can protect against future techniques, not just known ones from the past.

(R) Needless to say, state of the art steganography is usually fragile .... as part of the capacity needs to be reserved for the error correction.

(A) Current state-of-the-art steganography techniques are more robust than this, especially in regard to image secrets. Take for instance the most fragile of the steganography techniques used in this work---least significant bit (LSB) method. Changing a localized pixel, even many pixels in-fact, will have a small impact on the secret recovery performance. That is why just adding noise to an image is not effective at sanitizing an image. Text secrets, however, might be this fragile, but this too depends on the length of the message embedded. In regard to JPEG compression, this is a lossy image compression format. There are many cases where a lossless image file format is required, such as medical imaging or legal/forensic imaging.

(R) The experimental evaluation is poor...There is a lot of prior art in watermark removal. You should show that they do not work.

(A) If we understood correctly, you are asking why we haven't included works on watermark removal? If so, we have included watermark papers in our literature review (and added a few more as a suggestion from Reviewer 158U). The analysis in Section 2.2 helps to explain our experimental reasoning. For instance, many watermark removal techniques are not blind, as they require information from known hiding techniques. Additionally, while steganography and watermarking are related fields of work, we focus on comparisons to steganography, which only involves hidden information. Watermarking is an interesting field that includes both visible and invisible watermarks. We think comparisons between these two fields will be an interesting area to explore in the future. Thank you for your question!

[1] Hiding images in plain sight: Deep steganography.

[2] Steganographic generative adversarial networks.

[3] An embedding cost learning framework using gan.

[4] Automatic steganographic distortion learning using a generative adversarial network.

[5] Gan-based steganography with the concatenation of multiple feature maps.

[6] Hidden: Hiding data with deep networks.

[7] Sstegan: Self-learning steganography based on generative adversarial networks.

[8] Udh:Universal deep hiding for steganography, watermarking, and light field messaging.

We would first like to thank you for your review. We have addressed as many of your concerns as possible. Please see our responses below. (R) refers to you, the reviewer, and we have indicated our responses with an (A).

(R) "It appears that the noise variance parameter beta is left unspecified and only the time step parameter is identified, with no explanation of how it was set other than the ablation study."

(A)Thank you for pointing this out. This was an oversight on our part. We utilized the cosine beta scheduler from [1], which can be extended to any number of diffusion steps. The beta parameter is dependent on the time step parameters. We have updated this in the paper.

(R) "Steganography methods often have a strength parameter that can be varied, and it is not clear how important it is to match the diffusion strength to a particular steganography implementation and strength level. Choice of diffusion parameters may be critical to determine the level of steganography that can be sanitized, as evidenced by the ablation study."

(A) While steganography methods might have different strengths, DM-SUDS does not have to be tuned to a particular type of steganography strength. While we included an ablation study to see if there was a parameter setting where the sanitization capabilities of DM-SUDS breaks, tuning implies that the diffusion step parameter is sensitive to change, which is not the case.

(R)"It appears that the same dataset was used (CIFAR test set) for all of the experiments. If the parameters of the sanitization model (timestep t) were selected based on experiments performed using the same steganography methods and images, the method is technically not completely blind since the diffusion strength was tuned experimentally to the embedding method performance."

(A) We also include experiments with ImageNet in Section 5, Evaluation on ImageNet. Here we utilize the same arbitrarily chosen timestep value $t=250$ without any tuning.

(R) What was the value for beta, and how was timestep parameter t selected?

(A) For the beta value, we utilize a cosine scheduler from [1]. The initial timestep $t$ was arbitrarily chosen. Additional experiments were included regarding timestep $t$ to see if we could a find value where sanitization does not work.

(R) Page 8, "is a stronger hiding method" -> I would say more robust here, which is directly related to what is being measured (the fragility of the embedding). Stronger steganography implies strength in the steganographic sense and a measure of undetectability - something you are not measuring.

(A) Thank you for your suggestion! We have updated this in the paper.

(R) Could the strength of the diffusion be determined by estimating the amount of noise in the images in order to make this method completely blind?

(A) While this strategy would be useful if we wanted to find a lower bound for $t$ for each element, it is not necessary. We can over-approximate the number of timesteps for all hiding methods, removing this added calculation from the process.

(R) Please provide a table or plot of the performance metrics for the ablation studies.

(A) Thank you for this suggestion! We have updated this in the paper.

[1] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. arXiv preprint arxiv:2006.11239, 2020