Unleashing the Power of Deep Dehazing Models: A Physics-guided Parametric Augmentation Net for Image Rehazing

• Questions about idealized assumptions of the physical scattering model for haze removal.

Our method does account for the inaccuracies of the idealized PSM model. As detailed in Section 1 of the paper, the strong performance of PANet stems from its innovative two-step design. It employs a physically explainable (albeit simplified) model-based initial prediction, followed by a data-driven refiner that captures the residual errors from the idealized PSM model to mitigate the domain gap. The use of parametric augmentation in the ASM domain ensures the process remains both straightforward and physically interpretable. This approach represents a novel integration of model-driven and data-driven strategies.

• Effectiveness of PANet on real-world natural hazy images and the discussion of using hazy images generated by fog machine.

While hazes produced by fog machines may exhibit some differences from natural haze in real-world scenes, both types are governed by the same physical principles. Consequently, fog-machine-generated haze can effectively represent key characteristics of natural haze, such as its diffused, misty, and irregular nature, albeit with certain limitations compared to the complexity of real haze patterns. To address this gap, PANet employs a parametric augmentation scheme to diversify haze patterns, bridging the difference to some extent. As demonstrated in Tables 1 and 2, PANet enhances the performance of dehazing models not only on non-homogeneous haze datasets but also on the real-world natural haze dataset, RTTS. Additional qualitative results of dehazing models on the RTTS dataset are included in the supplementary materials.

• Qualitative results of full-size dehazed images.

We have included several full-size dehazed images of natural hazy scenes from the RTTS dataset in the supplementary materials. These examples demonstrate that PANet consistently delivers enhanced dehazing performance on real-world hazy images.

• Discussion of relying on existing datasets, the diversity of PANet, and the future plan.

This is an excellent point. While PANet significantly enhances the haze patterns in existing datasets of hazy-clean image pairs, thereby boosting the performance of current dehazing models, the diversity of the augmented dataset remains constrained by the environmental and lighting variations present in the original dataset, as noted by the reviewer. Addressing this limitation is crucial for further improving the comprehensiveness of augmented data.

To this end, we are expanding the capabilities of the physics-guided augmentation strategy by employing a novel approach that involves "disentangling" and "transferring" haze patterns from target-domain hazy images. Importantly, this process does not require paired haze-free versions of the images. Instead, it maps the haze patterns to the scene content of haze-free images in a different domain within the physics-parametric framework. We believe this haze-transfer capability, combined with parametric augmentation, will significantly enhance the spectrum of haze augmentation, enabling more effective domain adaptation.

Dear digo,

We have made every effort to address your concerns and sincerely hope our responses meet your expectations. We would greatly appreciate your feedback on our replies to help us further enhance the manuscript.

(1) While the hazy images in NH-Haze, generated by a fog machine, may not entirely replicate real-world natural haze, they effectively capture essential characteristics such as diffusion, mistiness, and irregularity since the fog is governed by physics laws. This makes them significantly more representative of natural haze compared to hazy patterns generated solely by the physical scattering model (PSM). Moreover, the enriched haze patterns achieved through our parametric augmentation methods---such as variations in thickness, density, and spatial distribution---further reduce the disparity between fog-machine-generated haze and real haze.

(2) Although our parametric augmentation is based on the PSM parametric domain, the subsequent Data-driven Haze Refinement (DHR) module bridges the gap between ASM-approximated haze and real haze, as detailed in our manuscript and responses.

We have incorporated additional evaluation metrics, including BRISQUE, MUSIQ, and PAQ2PIQ, for the RTTS and Fattal's datasets, as presented in Table 2. However, since these quality assessment metrics are not specifically designed for image dehazing, they may not accurately reflect dehazing performance. Therefore, we further conducted a user study on the RTTS dataset, detailed in Appendix A.6, showing that PANet-enhanced results received 63 of the preference votes. Additionally, we provide extensive qualitative comparisons of dehazing models in real-world natural scenes, as shown in Figures 14 to 22. These experiments highlight PANet's effectiveness in handling hazy images in real-world natural scenes.

• Contributions of PANet.

In this paper, we observe that effective data augmentation methods for real-world image dehazing have been underexplored. To address this gap, we propose PANet, an effective and robust haze augmentation method. D4 [1], a GAN-based approach, relies on unpaired synthetic hazy images to extract global haze coefficients. However, GANs often suffer from unstable training processes, lack of robustness, and difficulty in controlling pixel-wise haze conditions. Moreover, the limited training pairs provided by existing real-world dehazing datasets make it challenging to effectively optimize GAN-based methods.

In contrast, PANet is a physics-guided parametric augmentation network that automatically disentangles pixel-wise haze coefficients without relying on haze coefficient labels or GAN structures. We introduce an innovative approach by projecting haze coefficients into a parameter space, implicitly characterizing factors such as atmospheric light and haze density. This allows us to sample diverse haze conditions to augment hazy images not present in the training set. We believe that this implicit and physically guided process has the potential to advance the field of real-world image dehazing and inspire new research directions.

• Comparisons between PANet and existing data augmentation methods via Dehamer and DW-GAN.

Thank you for the suggestion. As the experiments take time to finish, we will provide additional comparisons with Dehamer and DW-GAN within a few days.

• Revision of the manuscript in L237 and Figures 2 and 3.

We have merged Figures 2 and 3 and fixed typos and reference errors in the updated paper.

• Effectiveness of PANet on RTTS for object detection.

Thank you for the suggestion. We will evaluate the impact of PANet-enhanced dehazing on the performance of downstream object detection on RTTS and provide the result soon.

• Experiments on a larger number of augmented data pairs.

Thank you for the suggestion. We will generate additional augmented images and provide experiments on a larger number of augmented data pairs within a few days.

[1] Self-augmented unpaired image dehazing via density and depth decomposition. In CVPR, 2022.

Dear BjsZ,

We have made every effort to address your concerns and sincerely hope our responses meet your expectations. We would greatly appreciate your feedback on our replies to help us further enhance the manuscript.

(1) We have conducted additional experiments to evaluate the convergence of dehazing performance relative to the number of augmentations. Additionally, we performed experiments using a larger number of augmented data pairs ($800\%$). The results show that PANet converges when using $600$ augmented data.

(2) Per your suggestion, We have conducted experiments to evaluate the impact of dehazing on downstream object detection using YoloV8 as the detection model on RTTS. However, we observed that the ground-truth bounding boxes in the RTTS dataset are incomplete, leading the detection model to predict results that appear more accurate than the ground truth. As a result, the AP scores can not accurately reflect the dehazing performance. Due to time constraints, we have not included visualization results of object detection in the Appendix. We will provide these results after publication.

Table A. Cross-dataset comparisons of dehazing performance of haze augmentation methods, including RIDCP, D4, PTTD, and PANet.

• Effects of pre-trained depth estimation network and the proposed Depth-Refinement Network (DRM).

We address this comment from two perspectives:

Depth Map Estimation: Our primary objective is not to estimate precise depth maps but rather to produce depth maps suitable for guiding realistic haze generation. Recognizing the inherent domain gap in any pre-trained depth estimator, we introduce a Depth Refinement Module (DRM) to mitigate this gap. Additionally, depth labels for the clean images in haze-clean image pairs are typically unavailable. To address this, we supervise the learning of the DRM using the discrepancy between the input hazy image and the regenerated hazy image, while keeping the ASM parameter maps unchanged in the "learning" mode.

Fair Comparison: To ensure a fair comparison, we adopt the same depth estimator, RA-Depth [1], as used in RIDCP [2].

• The choice of baseline dehazed methods and comparison with existing haze augmentation methods.

In fact, DW-GAN, Dehamer, and FocalNet are not primarily trained on synthetic haze data. These three SOTA models were explicitly designed for both synthetic and real-world image dehazing tasks and have been evaluated on real-world, non-homogeneous haze datasets.

Regarding the two methods you suggested, the authors of RIDCP [2] acknowledged its limitations in their paper, noting in the limitations section that “existing dehazing methods, including RIDCP, cannot process non-homogeneous (NH) haze well.” We attempted to re-train RIDCP using the NH-Haze20 training set; however, the model failed to converge and achieved a PSNR of only 12dB during training. Consequently, we chose not to include RIDCP as a baseline dehazing model in our experiments.

Additionally, DAD [3] is a CycleGAN-based haze-transfer method, with its hazing model being a UNet-like architecture not specifically designed for non-homogeneous (NH) haze. Regarding rehazing augmentation capabilities, D4 [4] has been reported in [4] to outperform DAD. However, we have already demonstrated that PANet consistently outperforms D4.

To further demonstrate the effectiveness of PANet, we perform cross-dataset validation comparisons, as shown in Table A. In this evaluation, dehazing models are trained on the baseline NH-Haze20 dataset and four expanded versions of NH-Haze20, augmented using PANet, RIDCP [2], D4 [4], and PTTD [5]. These models are then tested on four different datasets to assess their generalizability. The results indicate that the PANet-enhanced model consistently outperforms competing models across nearly all datasets, highlighting that PANet-augmented data effectively captures more diverse haze distributions.

• Discussion on artifacts in the qualitative results in Figure 7.

Due to the low resolution of these distant regions and the tendency for skies to be distorted by haze, fully recovering fine details remains challenging. Nevertheless, PANet significantly enhances dehazing performance by effectively eliminating haze artifacts compared to the baseline.

• Questions about the continuity, range, and parameters for the parametric augmentation in PANet.

In PANet, both atmospheric light map $A$ and haze density $\beta$ are treated as continuous parameters, where $A$ ranges in [0,1] and $\beta$ ranges in [0, 7.6] with an average value of 2.18. For haze-free regions, we inverse $A$ which is inverted to $1-A$ and $\beta$ is sampled from the range [0.6, 1.25] to generate haze patterns.

• Experiments on additional real-world hazy images.

We will include additional results and evaluation scores for real-world hazy environments in the revised manuscript and supplementary materials.

• Some typos: e.g., Ln237.

Thank you for your feedback, we have fixed the typos..

[1] Ra-depth: Resolution adaptive self-supervised monocular depth estimation. In ECCV, 2022.

[2] RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors. In CVPR 2023.

[3] Domain adaptation for image dehazing. In CVPR 2020.

[4] Self-augmented unpaired image dehazing via density and depth decomposition. In CVPR, 2022.

[5] Prompt-Based Test-Time Real Image Dehazing: A Novel Pipeline. In ECCV 2024.

Dear oeYm,

We have made every effort to address your concerns and sincerely hope our responses meet your expectations. We would greatly appreciate your feedback on our replies to help us further enhance the manuscript.

If the precise depth maps are not mandatory, why not directly utilize image $I_H$ to estimate the depth? The whole pipeline will be more flexible by adopting this strategy.

Let me specify additional results and other metrics. Other real-world hazy datasets like Fattal's dataset from [1], and other no-reference image quality assessment metrics like MUSIQ, HyperIQA, paq2piq, etc.

[1] Dehazing Using Color-Lines, TOG 2014

We have incorporated additional evaluation metrics, including BRISQUE, MUSIQ, and PAQ2PIQ, for the RTTS and Fattal's datasets, as presented in Table 2. However, since these quality assessment metrics are not specifically designed for image dehazing, they may not accurately reflect dehazing performance. Therefore, we further conducted a user study on the RTTS dataset, detailed in Appendix A.6, showing that PANet-enhanced results received 63 $\%$ of the preference votes. Additionally, we provide extensive qualitative comparisons of dehazing models in real-world natural scenes, as shown in Figures 14 to 22. These experiments highlight PANet's effectiveness in handling hazy images in real-world natural scenes.

We employ the Data-driven Refinement Module (DRM) to address the domain gap between the pre-trained depth estimator and the haze-free images in the NH-Haze20 training set. Since our objective is to augment the existing hazy image dataset (NH-Haze20), we exclusively use the haze-free images provided within this dataset. The augmented dataset is then utilized to train dehazing models, which are subsequently evaluated across other datasets.

Additionally, a refined depth map produced by DRM is illustrated in Figure 10 of the Appendix. Rather than aiming for precise depth estimation (as ground-truth depth maps are unavailable), DRM adjusts the depth map generated by the pre-trained depth estimator to make it "more suitable for real-haze generation." This refinement minimizes the discrepancy between the generated hazy image and its ground truth. As shown in Figure 2, the DRM learning process is supervised by the difference between the input hazy image and its regenerated version in the "learning mode" with the PSM parameter maps remaining unchanged.

Our experiments demonstrate that this data-driven depth refinement effectively addresses the highly ill-posed problem of transmission map (or depth map) estimation, improving the fidelity of cyclic real-haze regeneration in the learning mode. In Table 4, we compare PANet against two variants: PANet without the depth estimator and DRM ("w/o depth and DRM") and PANet without DRM ("w/o DRM"). The results clearly indicate that PANet with DRM achieves the best performance.

• Discussion on the paper: Structure Representation Network and Uncertainty Feedback Learning for Dense Non-Uniform Fog Removal [4]

Thank you for bringing this interesting work to our attention. We have cited this paper and explore how this real-world dataset could contribute to improving dehazing or potentially support future extensions beyond dehazing in the updated paper.

• Issue of colour shift in Figure 1 and 6.

Dehazing images with thick haze is a challenging inverse problem, as scene content is often heavily occluded by haze, and certain image regions may appear overexposed. Due to insufficient training data, existing state-of-the-art (SOTA) dehazing models frequently produce results with unsatisfactory visual quality and noticeable artifacts. While PANet-enhanced results may still exhibit slight color shifts compared to the ground truth, they are significantly cleaner and more visually appealing than those produced by SOTA baselines, demonstrating a substantial improvement.

• Dehazed results for the sky, white object, and road.

These regions are often overexposed and exhibit low contrast, providing insufficient visual cues for dehazing models to achieve effective restoration. The problem is exacerbated by limited training data, as the scarcity of diverse examples restricts the ability of dehazing models to learn effectively. PANet addresses this issue to some extent by generating diverse haze patterns, ranging from thick to thin haze, over these overexposed regions, as evidenced by the results.

[4] Structure Representation Network and Uncertainty Feedback Learning for Dense Non-Uniform Fog Removal. In ACCV 2022.

Table A. Cross-dataset comparisons of dehazing performance of haze augmentation methods, including RIDCP [1], D4 [2], PTTD [3], and PANet.

• Importance and challenges of non-uniform image dehazing and PANet’s approach to handling thick haze.

As highlighted in Table 1, existing SOTA dehazing methods struggle to handle non-uniform haze, resulting in subpar performance. The authors of RIDCP [1] also acknowledged this limitation in their paper (see its "limitations" section), stating that “existing dehazing methods, including RIDCP, cannot process non-homogeneous haze well.” This underscores the ongoing challenge of effectively addressing non-uniform image dehazing.

In Figure 6, we showcase dehazed images under thick haze conditions. The "Baseline" model (i.e., a SOTA model trained on the baseline dataset) fails to perform adequately in these scenarios, whereas PANet-enhanced models demonstrate significant improvements. PANet effectively augments hazy images containing thick haze, as illustrated in Figures 5(b) and 5(d). These augmented images expand the training datasets with more diverse haze conditions, leading to superior dehazing performance in challenging environments.

To further demonstrate the effectiveness of PANet, we perform cross-dataset validation comparisons, as shown in Table A. In this evaluation, dehazing models are trained on the baseline NH-Haze20 dataset and four expanded versions of NH-Haze20, augmented using PANet, RIDCP [1], D4 [2], and PTTD [3]. These models are then tested on four different datasets to assess their generalizability. The results indicate that the PANet-enhanced model consistently outperforms competing models across nearly all datasets, highlighting that PANet-augmented data effectively captures more diverse haze distributions.

• Qualitative comparisons between PANet and existing haze augmentation methods.

Thanks for the comment. Besides the qualitative comparisons between the PANet-enhanced model and the baseline, We have provided additional quantitative comparisons with exiting haze augmentation in Table A. We also provide the qualitative comparisons between PANet and existing haze augmentation methods in the section Appendix A.2.

• Difference between PANet and existing methods and additional comparison with a non-deep learning method

In our paper, we compared several representative haze augmentation methods, including PSM-based synthesis, PSM with adverse lighting conditions (RIDCP [1]), and the CycleGAN-based D4 [2], demonstrating that PANet significantly outperforms these approaches. We appreciate the reviewer for highlighting the recent ECCV 2024 non-deep learning method, PTTD [3], which generates visual prompts to enhance dehazing performance. However, as PTTD was introduced the same week as our ICLR submission, we were unable to include a comparison in the initial paper.

That said, as shown in Figure 8 in the updated paper, the visual prompts generated by PTTD often exhibit patch-wise artifacts that deviate significantly from real-world haze distributions. Consequently, applying the augmented haze patterns produced by PTTD to train dehazing models yields poor performance in real haze scenarios, achieving only 8–10 dB (see Table 1 in [3]). Therefore, we respectfully disagree with the claim that existing haze augmentation methods achieve performance comparable to PANet.

As detailed in the paper, the strong performance of PANet stems from its innovative two-step design. First, it employs a physically explainable (albeit simplified) model-based initial prediction, followed by a data-driven refiner that captures the residual errors from the simplified PSM model. The use of parametric augmentation in the ASM domain ensures the process remains both straightforward and physically interpretable. This approach is not merely a combination of existing methods but represents a novel integration of model-driven and data-driven strategies.

[1] RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors. In CVPR 2023.

[2] Self-augmented unpaired image dehazing via density and depth decomposition. In CVPR, 2022.

[3] Prompt-Based Test-Time Real Image Dehazing: A Novel Pipeline. In ECCV 2024.

Dear WXTj,

We have made every effort to address your concerns and sincerely hope our responses meet your expectations. We would greatly appreciate your feedback on our replies to help us further enhance the manuscript.

• Difference between PANet and CycleGAN.

As mentioned in section 1, PANet dose not rely on CycleGAN, which is known for its instability and inability to produce controllable results. In contrast, PANet is a robust haze augementation network that enables pixel-wise adjustments of haze conditions to augment realistic hazy images not provided in the training set.

• Discussion of dataset limitations.

To ensure a fair comparison, we use the NH-Haze20 training set to optimize both PANet and the dehazing models. To verify the generalizability of PANet, we perform cross-dataset evaluations by testing dehazing performance on the NH-Haze21, O-Haze, and I-Haze datasets. Additionally, we evaluate dehazing performance on the RTTS dataset, which contains 4,322 hazy images collected from real-world scenarios without corresponding haze-free images. These experiments demonstrate the generalizability and effectiveness of PANet.

• Discussion of computational footprint and plan to scale to larger datasets

We provide computational footprint of PANet in the section Appendix. PANet consists of 3M parameters and requires 23 GFLOPs, with an inference time of 25 ms for 256×256 images. In PANet, we focus on increasing haze diversity by utilizing haze-free images from the NH-Haze20 training set. Since PANet can be successfully optimized with limited training data, it has the potential to be applied to larger datasets for further augmentation. However, as existing real-world image dehazing datasets only provide limited training samples, we are unable to conduct experiments using larger datasets.

• Writing issues in line 238.

Thank you for your feedback, we have revised it.

• Dehazing performance for outdoor and indoor hazy images.

In PANet, we augment training pairs using outdoor haze-free images from the NH-Haze20 dataset, resulting in greater performance improvements on outdoor hazy images compared to indoor hazy images.

• Dehazing models in Figures 6 and 7.

We adopt dehazing models, including DE-GAN, DeHamer, and FocalNet, as listed on the left side of Figures 6 and 7.

• Discussion of using pre-trained depth estimator.

We address this comment from two perspectives:

Depth Map Estimation: Our primary objective is not to estimate precise depth maps but rather to produce depth maps suitable for guiding realistic haze generation. Recognizing the inherent domain gap in any pre-trained depth estimator, we introduce a Depth Refinement Module (DRM) to mitigate this gap. Additionally, depth labels for the clean images in haze-clean image pairs are typically unavailable. To address this, we supervise the learning of the DRM using the discrepancy between the input hazy image and the regenerated hazy image, while keeping the ASM parameter maps unchanged in the "learning" mode.

Fair Comparison: To ensure a fair comparison, we adopt the same depth estimator, RA-Depth [1], as used in RIDCP [2].

• The choice of baseline dehazed methods and their performance on the RTTS dataset.

We adopt state-of-the-art image dehazing models for experiments. Since RTTS consists of hazy images collected from natural scenes without corresponding haze-free images, existing dehazing methods struggle to fully recover high-quality clean images. However, PANet significantly improves baseline dehazing performance, producing visually more satisfactory results.

• Discussion of overfitting.

PANet can generate diverse hazy images to augment existing datasets, helping to prevent dehazing models from overfitting. As shown in Tables 1 and 2, dehazing models without PANet tend to overfit on the training sets, leading to unsatisfactory performance on the NH-Haze21, O-Haze, I-Haze, and RTTS datasets. In contrast, PANet-enhanced dehazing models consistently improve upon baseline models, delivering better dehazing results.

• Evaluation on the SOTS-Outdoor and SOTS-Indoor datasets.

Since PANet focuses on improving dehazing performance on real-world dehazing datasets, we do not use the SOTS-Outdoor and SOTS-Indoor datasets, as they consist of homogeneous synthetic haze.

• Evaluation under challenging conditions.

NH-Haze20 and NH-Haze21 datasets contain hazy images with thick haze. PANet consistently improve dehazing performance on these two datasets. We have demonstrate qualitative comparisons under thick haze in Figure 6.

[1] Ra-depth: Resolution adaptive self-supervised monocular depth estimation. In ECCV, 2022.

[2] RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors. In CVPR 2023.

PANet without reverse haze augmentation reduces the haze diversity in the augmented images, resulting in worse performance compared to the final version of PANet.

We have included additional evaluation metrics, including BRISQUE, MUSIQ, and PAQ2PIQ, for the RTTS and Fattal's datasets in Table 2. To further demonstrate PANet's capability in improving dehazing performance in real-world natural scenes, we conducted a user study on the RTTS dataset, detailed in Appendix A.6, showing that PANet-enhanced results received 63% of the preference votes. Additionally, we provide extensive qualitative comparisons of dehazing models in real-world natural scenes, as shown in Figures 14 to 22. These experiments highlight PANet's effectiveness in handling hazy images in real-world natural scenes.