Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects

Q3: Verify the generalizability of Camouflageator on the COD methods with Res2Net50 and Swin backbones.

We further assess the generalizability of Camouflageator on detectors with various backbones, including Res2Net50-based, Swin Transformer-based, and PVT V2-based detectors. We consider two conditions: (1) the backbone of the auxiliary generator adopts ResNet50; (2) the backbone of the auxiliary generator aligns with the detector. If the original method is denoted as ICEG, we term the methods optimized in condition 1 and condition 2 as ICEG+ and ICEG++, respectively. Results presented in the table reveal that Camouflageator consistently enhances the performance of these detectors in both conditions, with greater improvement observed when the generator shares the same backbone with the detector. Specifically, Camouflageator improves ICEG by 2.9%, 3.2%, and 4.3% when the backbone of the generator and the detector are Res2Net50, Swin Transformer, and PVT V2. Furthermore, Camouflageator enhances BGNet, a state-of-the-art Res2Net50-based detector, by 2.3% (BGNet+ with ResNet50-based generator) and 2.7% (BGNet++ with Res2Net50-based generator). For the Swin Transformer-based detector, Camouflageator improves ICON by 1.5% (ICON+ with ResNet50-based generator) and 3.7% (ICON++ with Swin Transformer-based generator). Additionally, HitNet, a PVT V2-based detector, exhibits improvements with our Camouflageator, showing an increase of 0.9% (HitNet+ with ResNet50-based generator) and 3.9% (HitNet++ with PVT V2-based generator).

Thanks for the valuable comments.

Q1: Descriptions of the quantity or quality of the synthesized camouflaged objects.

Quantitative analysis: There are three critical aspects to consider in maintaining the undetectability of a camouflaged object: (1) internal consistency (IC); (2) edge disruption (ED); (3) environment adaptation (EA). To assess the camouflaged level, accounting for color and contrast, we have designed three corresponding metrics:

$IC = \|\mathbf{y} \otimes \mathbf{x}_g-P_o^{I}\|^2,$

$ED = \| \mathbf{y}_e \otimes \mathbf{x}_g - P_e^{I} \|^2,$

$EA = \|\mathbf{x}_g-P_g^{I}\|^2,$

where $\mathbf{x}_g$ is the synthesized camouflaged image, $\mathbf{y}$ is the ground truth binary mask, $\otimes$ denotes element-wise multiplication, $\mathbf{y}_e$ is the weighted edge mask dilated by Gaussian function. $P_o^{I}$ is the image-level object prototype, averaging foreground pixels. $P_e^{I}$ is the image-level edge prototype, averaging edge pixels specified by $\mathbf{y}_e$. $P_g^{I}$ is the image-level object prototype, averaging global pixels.

For clarity and comparability, we normalize these three metrics between [0,1] and smaller values indicate more covert camouflaged objects. It's essential to note that the generation of additional camouflaged objects occurs exclusively on the training set. Consequently, we evaluate the enhanced camouflaged images and the original camouflaged images solely on the training set. As shown in the table, the enhanced camouflaged images generated by our Camouflageator are more deceptive than the original images.

Qualitative analysis: As shown in Fig. 6 in the manuscript, our Camouflageator can generate more concealed objects by hiding discriminative information. For example, the mouth and eyes of the fish are deliberately hidden in our synthesized image, and the physiological information of the rabbit is also weakened.

Q2: Visualization of module ablations.

We have included visualization maps of module ablations in Fig. S2 of the supplementary material. To enhance clarity, we have updated this figure using the CAM-family for better illustration. As exhibited in Fig. 2(d), the detector with CFC module can better excavate the internal coherence of camouflaged objects, facilitating the generation of more complete prediction maps. Additionally, as depicted in Fig. 2(e), the network equipped with the ESD component can decrease the uncertainty fringes and thus generate more precise results with clearer boundaries.

Dear Reviewer pcqB,

We appreciate the reviewer’s time and effort in reviewing our manuscript and insightful comments.

As the closure of the discussion period is approaching, we would like to bring the review’s attention and check if the reviewer could let us know whether the concerns or the misunderstanding have been addressed.

If this is the case, we would appreciate if you could adjust your rating accordingly.

Best regards,

Authors

Q6: Why resnet50 rather than transformer?

We choose ResNet50 as the backbone of the auxiliary generator, aligning with common practice in generator design. It's important to note that our Camouflageator framework is not confined to specific models. We conducted a thorough evaluation of Camouflageator's generalizability across generators with diverse backbones, including Res2Net50-based, Swin Transformer-based, and PVT V2-based generators. In exploring the suitability of different backbones for Camouflageator, our experiments focused on two conditions: (1) the auxiliary generator's backbone is ResNet50, and (2) the auxiliary generator shares the same backbone with the detector. Denoting the original method as ICEG, we term the methods optimized in conditions 1 and 2 as ICEG+ and ICEG++, respectively. The results presented in the table consistently demonstrate that Camouflageator enhances the performance of these detectors under both conditions, with more significant improvement observed when the generator shares the same backbone with the detector. Specifically, Camouflageator improves ICEG by 2.9%, 3.2%, and 4.3% when the backbones of the generator and the detector are Res2Net50, Swin Transformer, and PVT V2, respectively. Furthermore, Camouflageator enhances BGNet, a state-of-the-art Res2Net50-based detector, by 2.3% (BGNet+ with ResNet50-based generator) and 2.7% (BGNet++ with Res2Net50-based generator). For the Swin Transformer-based detector, Camouflageator improves ICON by 1.5% (ICON+ with ResNet50-based generator) and 3.7% (ICON++ with Swin Transformer-based generator). Additionally, HitNet, a PVT V2-based detector, shows improvements with our Camouflageator, exhibiting an increase of 0.9% (HitNet+ with ResNet50-based generator) and 3.9% (HitNet++ with PVT V2-based generator).

Thanks for the valuable comments. If not specifically stated, all experiments are conducted on the COD10K dataset for space limitation.

Q1: Difference with CamDiff.

Idea: The motivation behind CamDiff is to address the challenge of existing COD methods in distinguishing between salient and camouflaged objects. Conversely, our approach focuses on enhancing the discriminative feature extraction capacity of current detectors to improve their overall generalizability. Since the root issue CamDiff aims to resolve is the insufficient discriminative information extraction by existing detectors between salient and camouflaged objects, our strategy can also effectively address this problem.

Implementation: CamDiff generates synthetic salient objects within camouflaged scenes and establishes a fixed dataset for detector training. In contrast, our Camouflageator employs a continuously updated generator, synthesizing training data while simultaneously training the detector in an adversarial manner. This approach tailors the dataset for each detector, enhancing flexibility and ensuring superior performance.

Experiment: The results presented in the table compare detectors trained with our Camouflageator and CamDiff. Notably, detectors optimized by Camouflageator consistently achieve superior performance. In contrast, detectors optimized by CamDiff occasionally exhibit lower performance than their original versions. This discrepancy may stem from the introduction of salient objects disrupting overall image harmonization during CamDiff's dataset construction. Moreover, the second row of Fig. S2 in the supplementary material illustrates that the original detector, ICEG, encounters challenges in separating salient and camouflaged objects. However, after being optimized by the Camouflageator, ICEG+ adeptly identifies camouflaged objects. This underscores that Camouflageator effectively addresses the problem CamDiff seeks to solve, yielding substantial performance improvements.

Q2: More related works.

We add the discussions of those papers in the Related Works. Notice that our related work provides a comprehensive categorization of the types of learning-based methods available and that the three learning-based methods can also be classified into these three categories. For instance, MGL [1] falls into the category utilizing a joint training strategy. Similarly, UGTR [2] and DTAF [3] can be classified under the category employing multi-scale feature aggregation.

Q3: Losses used for segmentation.

It is a common practice to use weighted BCE loss and weighted IoU loss for supervision in the field of COD, and our approach aligns with this convention. Weighted BCE loss treats each pixel as an independent sample for prediction, whereas weighted IoU loss provides a more comprehensive evaluation of the final prediction. The synergy between these two loss functions ensures a thorough assessment of the prediction results.

Q4: Difference between camouflaged consistency loss and contrastive loss.

The contrastive loss emphasizes the alignment of positive pairs (and misalignment of negative pairs). As a difference, our camouflaged consistency loss simultaneously encourages intra-group compactness and inter-group separation. Our loss is particularly designed for camouflaged image pixels. In this context, foreground pixels exhibit a strong intrinsic correlation, while the composition of background pixels tends to be intricate. As a result, our camouflaged consistency loss exclusively constrains the foreground pixels. We aim for the foreground pixels to exhibit compactness among themselves, and we further desire these compact foreground pixels to be distinct from the background pixels. Notably, we intentionally avoid enforcing compactness on background pixels due to their inherent redundancy, as such a requirement could compromise the feature extraction capability.

Q5: Are $\mu$ and $\sigma$ the same?

$\mu$ and $\sigma$ are variational parameters and they are different. We reformulate Eq. 17 to avoid misunderstanding.

[1] MGL, CVPR21.

[2] UGTR, ICCV21.

[3] DTAF, TCSVT.

Dear Reviewer bgwW,

We appreciate the reviewer’s time and effort in reviewing our manuscript and insightful comments.

As the closure of the discussion period is approaching, we would like to bring the review’s attention and check if the reviewer could let us know whether the concerns or the misunderstanding have been addressed.

If this is the case, we would appreciate if you could adjust your rating accordingly.

Best regards,

Authors

Thanks for the valuable comments.

Q1: ICEG fails to identify the foreground object when concealed objects have very similar structural information to the background.

Accurately separating foreground objects from backgrounds with very similar structure information is a common challenge for almost all COD solvers [1,2,3]. To overcome this problem, we propose the adversarial Camouflageator framework along with the CFC and ESC modules proposed in ICEG, achieving results that are both qualitatively (Fig. 5 in the manuscript and Fig. S1 in the supplementary material) and quantitatively (Table 1 in the manuscript and Table S1 in the supplementary material) better than existing methods.

While being more favorable than existing COD solvers for this challenging case, we observe that our method may encounter challenges in accurately identifying the foreground, particularly in extreme scenarios, as depicted in Fig. S3 in the supplementary material. We attribute this to the intrinsic structural similarity between the foreground and the background. In response, as a future work, we are considering extending our method in the following ways to address this issue:

• We intend to employ a coarse-to-fine framework, wherein we can first detect those candidates and then finetune the coarse maps to ensure accurate segmentation.
• We propose to design a multi-scale feature grouping module that first groups features at different granularities and then aggregates these grouping results. Grouping similar features encourages detection coherence, helping obtain accurate results for both single and multiple-object images.

We have incorporated the above content into the "Limitations" section of the supplementary material.

Q2: Many false positive detection results are outputted.

As demonstrated in the visual verification presented in Fig. S2 in the supplementary material, the incorporation of our proposed ESD module effectively reduces both false positive and false negative regions. Furthermore, our method exhibits superior performance in both qualitative and quantitative analyses. The results in Fig. 5 in the manuscript and Fig. S1 in the supplementary material underscore our method's effectiveness in addressing the false positive problem compared to existing methods.

Nonetheless, when confronted with the extreme structural similarity challenge mentioned in Q1, our method may still encounter some false positive regions. To address this challenge in the future, we will explore and implement the strategies discussed in Q1.

[1] PreyNet, ACM MM22.

[2] FGANet, NeurIPS23.

[3] FEDER, CVPR23.

Dear Reviewer fgyc,

We appreciate the reviewer’s time and effort in reviewing our manuscript and insightful comments.

As the closure of the discussion period is approaching, we would like to bring the review’s attention and check if the reviewer could let us know whether the concerns or the misunderstanding have been addressed.

If this is the case, we would appreciate if you could adjust your rating accordingly.

Best regards,

Authors

Q4: Verify the generalization of the proposed Camouflageator.

We have proved the generalizability of our Camouflageator, whose generator employs ResNet50 as the backbone, on several cutting-edge ResNet50-based detectors, including PreyNet, FGANet, and FEDER, in Table 1 in the manuscript. Camouflageator consistently enhances other detectors by 2.8% (PreyNet), 2.2% (FGANet), 2.3% (FEDER). Besides, Table S4 in the supplementary material illustrates that the network trained with Camouflageator excels in handling extreme conditions, such as small objects and multiple objects.

We further assess the generalizability of Camouflageator on detectors with various backbones, including Res2Net50-based, Swin Transformer-based, and PVT V2-based detectors. We consider two conditions: (1) the backbone of the auxiliary generator adopts ResNet50; (2) the backbone of the auxiliary generator aligns with the detector. If the original method is denoted as ICEG, we term the methods optimized in condition 1 and condition 2 as ICEG+ and ICEG++, respectively. Results presented in the table reveal that Camouflageator consistently enhances the performance of these detectors in both conditions, with greater improvement observed when the generator shares the same backbone with the detector. Specifically, Camouflageator improves ICEG by 2.9%, 3.2%, and 4.3% when the backbone of the generator and the detector are Res2Net50, Swin Transformer, and PVT V2. Furthermore, Camouflageator enhances BGNet, a state-of-the-art Res2Net50-based detector, by 2.3% (BGNet+ with ResNet50-based generator) and 2.7% (BGNet++ with Res2Net50-based generator). For the Swin Transformer-based detector, Camouflageator improves ICON by 1.5% (ICON+ with ResNet50-based generator) and 3.7% (ICON++ with Swin Transformer-based generator). Additionally, HitNet, a PVT V2-based detector, exhibits improvements with our Camouflageator, showing an increase of 0.9% (HitNet+ with ResNet50-based generator) and 3.9% (HitNet++ with PVT V2-based generator).

Thanks for the valuable comments.

Q1: Missing primary storyline and lacking the establishment of the relationship between Camouflageator and ICEG.

As stated in the third paragraph of the Introduction, our method is inspired by the prey-vs-predator game, which leads to ever-strategic preys and ever-acute predators, and we propose to address COD by developing algorithms on both the prey side (Camouflageator) that generates more deceptive camouflage objects and the predator side (ICEG) that produces complete and precise detection results. Therefore, Camouflageator and ICEG are complementary and can be combined.

In Section 3.2.4, we present how to integrate the Camouflageator framework with ICEG and denote the integrated method as ICEG+. As depicted in Fig. 1 in the manuscript, ICEG+ successfully achieves the overarching objectives of our paper and excels in addressing the extreme challenges in COD, including false localization, incomplete prediction, and ambiguous boundaries.

In the updated manuscript, we have revised the Introduction to make it clearer about the relationship between Camouflageator and ICEG. Thank you for pointing it out.

Q2: Difference of Camouflageator between the adversarial training framework in other tasks.

Our method diverges significantly from practices employed in other tasks, such as image deblurring, both in fundamental idea and implementation.

Fundamental idea: Existing methods, such as image deblurring methods [1,2,3], typically utilize adversarial training strategies, employing a classification-based discriminator to ensure the generation of visually faithful results. The focus lies on the generator side. In contrast, drawing inspiration from the prey-vs-predator game, we introduce a task-oriented framework, Camouflageator, for COD. The primary objective is to generate more deceptive camouflage objects, thereby enhancing the generalizability of the detector (termed task-oriented). Our focus lies on the detector (or discriminator) side.

Implementation: To achieve the goal of deception, we introduce the concealment loss (Eq. 3) for generating deceptive objects, a concept not previously proposed in existing adversarial training methods. Unlike purely data-driven generators (task-agnostic), our auxiliary generator is partially supervised by the detector (Eq. 4) in a task-wise manner. Additionally, in contrast to prevailing adversarial training strategies, we deliberately modify the network and omit the discriminator (refer to Table S11 in the supplementary material). This choice which is deliberate as the objective of our Camouflageator framework is to bolster the generalizability of the detector, not to ensure complete unbiasedness of the synthesized images' distribution from the original camouflaged data. Overemphasis on the latter aspect may even lead to a degradation of detection performance.

Q3: Description of Fig. 3.

The input and output of Fig.3 are a camouflaged image and the corresponding prediction result generated by our ICEG. The proposed ICEG mainly has two critical components: the camouflaged feature coherence (CFC) and the edge-guided segmentation decoder (ESD). CFC employs the intra-layer feature aggregation (IFA), the contextual feature aggregation (CFA), and a camouflaged consistency loss $L_{cc}$ to alleviate incomplete segmentation. ESD comprises an edge reconstruction (ER) module and an edge-guided separated calibration (ESC) module to cooperatively eliminate ambiguous boundaries. We have revised the caption of Fig. 3 to make it easier to understand.

We have reorganized the Methodology to present clear and concise motivations and rationales for each design.

[1] Deblurgan, CVPR18.

[2] RWB, CVPR20.

[3] SAMD, TIP.

Q5: Ethics concerns.

Analysis and applications of our method: Inspired by the prey-vs-predator game, our method starts from the mimicry of animals and aims to exploit the potential value of zoology and thus contribute to species discovery. Our method aims to solve the segmentation task, termed COD, that aims to detect and segment objects seamlessly integrated into surrounding environments. Therefore, our method has various latent real-life applications [4], ranging from concealed defect detection in industry, pest monitoring] in agriculture, and polyp image segmentation in medical diagnosis, to recreational art and photo-realistic blending in art. However, our approach is not directly oriented to military scenarios because we are neither considering the complex scenarios of war nor do we involve special processing of human data, such as facial information. Note that our method can be applied to a large variety of different use cases which are beyond the scope of this work. A careful study of the data, model, intended applications, safety, risk, bias, and societal impact is necessary before any real-world application.

Potential implications of our method: We appreciate the reviewer’s advice and add an ethical statement to the manuscript.

[4] COD, CVPR20.

Dear Reviewer VfAG,

We appreciate the reviewer’s time and effort in reviewing our manuscript and insightful comments.

As the closure of the discussion period is approaching, we would like to bring the review’s attention and check if the reviewer could let us know whether the concerns or the misunderstanding have been addressed.

If this is the case, we would appreciate if you could adjust your rating accordingly.

Best regards,

Authors