RUN: Reversible Unfolding Network for Concealed Object Segmentation

Thanks for the valuable comments.

W1. Challenges in introducing DUNs and importance of modeling high-level vision tasks

In P3L142, we state that deep unfolding networks (DUNs) are underexplored in high-level vision tasks for lacking intrinsic models.

Challenges: DUNs rely on model-based optimization, which is effective in low-level vision tasks due to well-defined physical models (e.g., Retinex model for low-light enhancement, Atmospheric Scattering model for dehazing). However, high-level vision tasks lack an explicit physical model, limiting DUN applications.

Importance:

• Explicit constraints: Modeling high-level vision tasks enables task-specific constraints to guide optimization explicitly. For example, Eq. (4) introduces a residual sparsity constraint to refine segmentation and reduce uncertainties (P3L130, right part). This is more direct than implicit backpropagation. Table 5 shows a performance drop when removing this constraint (w/o prior $\hat{\mathbf{M}}_k$).
• Introducing DUNs into high-level vision: With an explicit model, DUNs integrate optimization with deep networks, balancing interpretability and generalizability (P2L58, right part). RUN benefits in effectiveness (Table 1), efficiency (see W2), and degradation resistance (Fig. 5) in COS.

W2. Efficiency analysis

The designs of $\hat{\mathcal{M}}(\cdot)$ and $\hat{\mathcal{B}}(\cdot)$ follow mathematical principles and may appear complex, but we only make fixed hyperparameters learnable and enforce weight sharing for SOFS and ROBE across stages. In the table, our method is more efficient than SOTAs.

W3. Comparison with more methods

BiRefNet [1] targets high-resolution segmentation (HRS); ZoomNeXt [2] focuses on multiple input scales (MIS), differing from our setting.

Higher resolution and multiscale inputs enhance subtle discriminative cues but add computational overhead. We also report our results in these settings.

For HRS, we retrain our method following BiRefNet [4], replacing our encoder with SwinL and training at 1024 x 1024 resolution. We evaluate our results on the three datasets reported in BiRefNet and find that RUN outperforms BiRefNet in most metrics. This is primarily due to our reversible modeling strategy.

For MIS, we compare with ZoomNeXt (352 × 352). In the table, RUN outperforms ZoomNeXt in 8 out of 12 metrics. Unlike ZoomNeXt’s specialized extraction-fusion strategy, RUN simply concatenates multiscale features. Adding ZoomNeXt’s strategy to RUN yields RUN++, improving results.

W4. Performance in segmentation refinement

RUN, an end-to-end network for COS, can also serve as a segmentation refiner by initializing $\mathbf{M}_0$ as the coarse mask.

We compare with several segmentation refiners: traditional methods (dense crf (DCRF) [3] and biliteral solver (BS) [4]) and learning-based methods (SegRefiner [5] and SAMRefiner [6]). To ensure fairness, we retrain the learning-based ones for COS, SegRefiner+ and SAMRefiner+, following their training rules.

Traditional methods fail to refine FEDER’s segmentation in COD10K due to the challenges of COS (objects blending with surroundings). SegRefiner and SAMRefiner also perform suboptimally with their provided models, as coarse COS masks contain segmentation errors and uncertain regions, complicating refinement.

Retraining improves SegRefiner+ and SAMRefiner+, with SAMRefiner+ getting results comparable to RUN. However, RUN (FPS: 22.75) outperforms SAMRefiner (FPS: 0.92) and SegRefiner (FPS: 0.89) in efficiency.

Refinement results for FGANet’s masks are discussed in W3 (reviewer 45Vr), with similar conclusions.

[1] BiRefNet, CAAI AIR24

[2] ZoomNext, TPAMI24

[3] Dense CRF, NIPS21

[4] Bilateral Solver, ECCV16

[5] SegRefiner, NIPS23

[6] SAMRefiner, ICLR25

Thanks for the valuable comments.

W1. More related works

In related works, the COS component focuses on the development of the reversible modeling strategy, while our coverage of DUN is limited due to the underexploration of this technique in high-level vision tasks. To provide a more comprehensive overview, we will add a section that systematically reviews the advancements in the COS field.

W2&W3. Why use a simple network in ROBE? How about replacing it with a more complex one?

(1) The reconstruction task in ROBE is relatively easy, compared to most low-level vision tasks, as the input images are of high quality. Thus, we select a lightweight network to balance performance and computational efficiency. Experimental results validate the effectiveness of this design choice.

(2) As shown in Table 5, replacing the simple network with more complex networks brings limited performance gains. However, the results in Fig. 5 indicate that a more complex network can better resist degradation conditions.

(3) Furthermore, as shown in the table, increasing the stage number can also improve the network’s degradation resistance capacity. These findings can help us further exploit the potential of this DUN-based framework.

W4. Performance of SOFS

As shown in the table, removing ROBE leads to a noticeable performance decline. The resulting model underperforms compared to the state-of-the-art method, FocusDiff, further highlighting the effect of our approach. This can also be verified by Table 11, where replacing SOFS with the core structures of existing methods results in clear performance improvements.

W5. Compare with bi-level optimization on more degradation scenes

(1) As shown in Fig. 5, replacing the original simple network with a more complex dehazing network leads to RUN+, which improves robustness against haze degradation. To further evaluate our approach, we test its performance within a bi-level optimization (BLO) framework following the setup of HQG-Net [1], where ROBE and SOFS are treated as low-level and high-level vision blocks, as illustrated in Fig. S2. Evaluating performance on COD10K using $F_\beta$ and $E_\phi$, we find that RUN+ consistently outperforms the BLO framework across different haze concentrations.

(2) Additionally, we further analyze the performance in two degradation scenarios that are extremely challenging for the COS task, i.e., the low-light and low-resolution scenarios.

Low-light scenarios deepen the concealment of objects by lowering color contrast, while low-resolution scenes directly reduce the discriminative cues by reducing the number of valid pixels.

As with the above content, we also simulate degradation on concealed images in COD10K.

For the low-light scenarios, we applied image darkening [2] and degraded the dataset into three levels: slight low-light, medium low-light, and severe low-light. We employ Reti-Diff [3] as the reconstruction network in the ROBE module. Experiments verify that our RUN framework achieves better performance than the BLO version.

For the low-resolution scenes, we use bicubic downsampling and select x2, x4, x8 for reconstruction and segmentation. DiffIR [4] is selected as the reconstruction network. As shown in the table, RUN still outperforms BLO in the low-resolution challenges.

To sum up, our RUN framework is not only a novel unfolding-based application for high-level vision tasks but also effectively integrates high-level and low-level vision, ensuring robust performance even in degraded scenarios.

[1] HQGNet, TNNLS23

[2] StableLLVE, CVPR21

[3] Reti-Diff, ICLR25

[4] DiffIR, ICCV23

Thanks for the valuable comments.

W1. Efficiency analysis

We compare the parameters, FLOPS, and FPS between our RUN against cutting-edge methods on three backbones. Our stage number is set to 4. As shown in the table, our method is more efficient across all three backbones with the input size of 352×352.

Besides, we analyze the computational cost of our RUN framework with varying stage numbers, along with their performance using ResNet50 as the backbone on COD10K. Since the SOFS and ROBE modules at different stages share the same weights, all versions of RUN maintain a consistent parameter count. As shown in the table, our method outperforms existing state-of-the-art methods once the stage number reaches 3. For an optimal balance between efficiency and performance, we set the stage number to 4.

W2. Ablation study: baseline and more results

As shown in Table 5, we conduct ablation studies on the ResNet50-based RUN framework.

To ensure the integrity of our reversible modeling strategy, we retain both SOFS and ROBE rather than performing breakdown ablations. Instead, in Table 5, we evaluate the contribution of each content within SOFS and ROBE by either replacing them with alternative designs or directly removing them, both of which lead to performance decreases.

Here we report the results of removing SOFS and ROBE. To remove the segmentation module SOFS, we add an extra segmentation head, i.e., the convolution blocks, after ROBE, with the segmentation loss unchanged. For the case w/o ROBE, we simply remove ROBE and keep SOFS. Besides, we include a baseline where both SOFS and ROBE are removed, leaving only the segmentation head in each stage. As shown in the table, adding SOFS and ROBE brings a clear performance gain.

W3. Comparison with segmentation refinement methods

Our RUN is an end-to-end network designed for COS. For its structural characteristics, it can also be applied for refining coarse masks by initializing $\mathbf{M}_0$ as the coarse mask.

We compare our method with segmentation refinement methods, including traditional methods (dense crf (DCRF) [1] and biliteral solver (BS) [2]) and learning-based methods (SegRefiner [3] and SAMRefiner [4]). We use segmentation results from FEDER and FGANet as coarse masks. For a fair comparison, we also provide retrained versions of the learning-based refiners for the COS task, SegRefiner+ and SAMRefiner+, following their original training rules.

As shown in the table, traditional methods fail to improve the segmentation results of FEDER and FGANet. SegRefiner and SAMRefiner also achieve suboptimal performance with their provided models.

This is because concealed object segmentation is an inherently difficult task, where objects are visually blended with their surroundings. Hence, unlike common segmentation tasks, this task provides far fewer discriminative cues for feature extraction. Besides, the coarse segmentation masks often suffer from significant quality issues, such as mis-segmentation, blurred edges, and high uncertainty in certain pixel regions, making mask refinement particularly challenging.

After retraining, the learning-based refiners show noticeable performance gains, with SAMRefiner+ achieving results comparable to our RUN. However, our RUN framework achieves an FPS of 22.75, outperforming SAMRefiner with only 0.92 FPS.

[1] Dense CRF, NeurIPS21

[2] The fast bilateral solver, ECCV16

[3] SegRefiner, NeurIPS23

[4] SAMRefiner, ICLR25

Thanks for the valuable comments.

W1. Why only conducting ablation studies on COD10K

Given that COD10K is a representative and high-quality dataset, we follow existing methods [1,2,3] to conduct ablation studies on COD10K. We have now included the results of ablation studies on three extra datasets. As shown in the table, the outcomes remain consistent with our initial conclusions. We will add this content to the revised manuscript.

W2. Efficiency, trade-off, and the best scenario of RUN

(1) Efficiency: We compare parameters, FLOPS, and FPS in three backbones and find that our RUN surpasses existing cutting-edge methods across all three backbones.

(2) Structure-level trade-off: Although different stages in RUN share the same weights, the stage number still affects the overall computational cost. To analyze the trade-off, we investigate the impact of the stage number on our ResNet50-based framework. In the table, our method surpasses existing methods when the stage number reaches 3. For an optimal balance, we set the stage number to 4.

(3) Framework-level trade-off: RUN achieves cutting-edge performance for the theoretical coupling of the segmentation module (SOFS) and the reconstruction module (ROBE). As shown in the table, ROBE brings an evident gain with limited costs. This can also be verified by Table 11, where replacing SOFS with the core structures of existing methods brings clear profits. Ablations in Table 5 indicate that replacing the simple reconstruction network with more complex ones yields marginal gains. Hence, we opt for a simple network here.

(4) The best application scenarios: RUN is not only a novel unfolding-based application for COS in segmenting clean concealed objects, but also has the potential to ensure robust performance in degraded scenes. As shown in Fig. 5, when replacing the reconstruction network with a more complex one, we get RUN+ and observe better hazy resistance capacity.

To further verify this, we compare with bi-level optimization (BLO), a degradation-resistant framework. The differences between the two frameworks can be seen in Fig. S2, where our framework is superior in the theoretical coupling of the two models. We retrain our network (RUN+) in the BLO framework and report the results on COD10K with $F_\beta$ and $E_\phi$. We find that our RUN+ surpasses BLO across different hazy concentrations.

Besides, we analyze performance in two other degradation scenes, i.e., the low-light and low-resolution scenes, and observe that our RUN framework consistently surpasses BLO. For space limitations, we put the results in the response of W5 of the reviewer MZkc. The reviewer can refer to the tables for more details.

[1] PFNet, CVPR21

[2] ZoomNet, CVPR22

[3] FEDER, CVPR23