Unconstrained Salient and Camouflaged Object Detection

Q3: Also, when evaluating, there can be some factors influencing the actual performance. For example, the different difficulty of the two tasks: if the model performs well at SOD but poor at COD, will it achieve the same score on CSCS with another model which performs averagely at both tasks?

Ans for Q3:

To analyze the impact of the network's performance bias between the detection of salient and camouflaged objects on CSCS, we performed a detailed data analysis of the methods presented in Table 2 of the manuscript. The results are shown in Table R2. In this analysis, Diff represents the difference value between $IoU_S$ and $IoU_C$. A smaller Diff indicates that the network treats the detection of the two attributes more equitably, with less bias toward either attribute. We arranged the table from left to right based on ascending Diff values. It can be observed that as Diff increases, CSCS does not exhibit a linear increase or decrease. This indicates that CSCS is an independent metric focused on the degree of attribute confusion, and it does not vary systematically with the performance bias between the two attributes.

Table R2: The Analysis of the Impact of Diff on CSCS. Adapter denotes SAM2-Adapter.

Q5: In the metric CSCS, I did not see the $P_{CB}$ and $P_{SB}$, which reflects the false positive rate to some extent. Can you explain the reason why $P_{CB}$ and $P_{SB}$ are not considered?

Ans for Q5:

The CSCS calculation formula consists of two components: the proportion of camouflage pixels incorrectly predicted as saliency among all pixels predicted as saliency, and the proportion of saliency pixels incorrectly predicted as camouflage among all pixels predicted as camouflage, which is inspired by IoU. The evaluation focuses on the model's ability to distinguish between saliency and camouflage pixels, specifically assessing the degree of confusion between these two attributes, without considering the salient and camouflaged pixels incorrectly predicted as background, i.e., $P_{SB}$ and $P_{CB}$.

Q4: Also, when evaluating, there can be some factors influencing the actual performance. Will the size [1] and number [2] of objects influence the evaluation? The authors are expected to conduct some discussion on the evaluation.

Ans for Q4:

Additionally, our USCOD benchmark adopts pixel-level evaluation metrics rather than instance-level metrics (like the Average Precision (AP) metric in the COCO benchmark) or sample-level metrics (like the F-measure widely used in the SOD and COD fields, which calculates the F-measure for each sample). Instead, we adopt an approach inspired by semantic segmentation benchmarks (such as Cityscapes, PASCAL VOC, and ADE20K) to comprehensively evaluate the model’s performance in recognizing saliency, camouflage, and background attributes.

Specifically, we calculate a confusion matrix across all pixels in the testing set, followed by calculating the IoU score for each attribute, as well as the mIoU, mAcc, and our CSCS metric. Therefore, the evaluation is not influenced by the size [1] or number [2] of objects. The Appendix of the manuscript provides the confusion matrix generated by our method on the CS12K testing set.

However, after carefully reviewing [1] and [2], we realized that the size and number of objects are crucial factors in evaluating Multi-object Salient Object Detection. Similarly, for our proposed USCOD task, this is equally important since multi-object scenarios are widely represented in our benchmark.

To analyze the impact of object size and number on the model's performance in USCOD, we adopted the metrics mentioned in [1] to evaluate our model's performance in detecting salient and camouflaged object. Additionally, we assessed the performance of the SOD model ICON and the COD model ICEG for comparison. The detailed results are shown in Table R3 and Table R4. It can be observed that, compared to the size-sensitive (e.g., $AUC↑$ and $F_m^{\beta}↑$) and size-invariance (e.g., SI-$AUC↑$ and SI-$F_m^{\beta}↑$) metrics, our method exhibits smaller performance fluctuations, demonstrating its robustness to variations in object size and number in the scene.

Table R3: Performance of detecting salient object on CS12K testing set.

Table R4: Performance of detecting camouflaged object on CS12K testing set.

To comprehensively evaluate these important metrics, we will incorporate them into the USCOD benchmark and continue assessing other models' performance on these metrics. Thank you for your valuable suggestion.

In the future, we also plan to explore an instance-level USCOD detection task, namely the Unconstrained Salient and Camouflaged Instance Segmentation (USCIS) task. The details of this task can be found in §G of the Appendix in the manuscript. For USCIS, the size-invariant metrics and size-invariant optimization loss mentioned in [1] are likely to be more suitable for evaluating and optimizing model performance. Once again, we sincerely appreciate your insights! If you have any further suggestions, we would be most grateful to hear them.

---

---

Reference:

[1] Li F, Xu Q, Bao S, et al. Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection. Proceedings of the 41st International Conference on Machine Learning, PMLR 235:28989-29021, 2024.

[2] Mingchen Zhuge, Deng-Ping Fan, Nian Liu, Dingwen Zhang, Dong Xu, and Ling Shao. Salient object detection via integrity learning. IEEE TPAMI, 45(3):3738–3752, 2022.

Q2: The loss function of the proposed method is not stated clearly. In equation (4), is there some difference when calculating loss for salient and camouflaged objects? If not, the different difficulties of COD and SOD may make the optimization less effective.

When evaluating and optimizing, do you give equal treatment to salient and camouflaged objects? If so, why not give different weights to the two types of objects considering their difference in difficulty?

Ans for Q2:

Considering that the number of images across the four scenes is balanced, we assumed the proportions of salient and camouflaged objects to be consistent. Therefore, we treated salient and camouflaged objects equally during evaluation and optimization. Since the network already achieved state-of-the-art performance on existing metrics, no further optimization was conducted.

However, your suggestion has been incredibly insightful. Inspired by your suggestion, we considered both the total pixel count and the difficulty of the two attributes, and attempted to address the issue by using Focal Loss. Specifically, we first calculated the pixel count ratio between salient and camouflaged objects, which is 6:4. Then, we set the weight ratio between saliency and camouflage attributes to 4:6, while keeping the default γ value of 2 to account for the varying difficulty of different pixels.

The experimental results using different loss functions are summarized in Table R1:

Table R1: The Experimental Results using Different Loss Functions.

From the table, we can observe that our model, when using Focal Loss, demonstrates potential, particularly in Scene C and in terms of CSCS. This suggests that Focal Loss may help optimize the model’s ability to distinguish between salient and camouflaged objects in complex scenes. However, no improvement in the IoU of Overall Scene was observed. We hypothesize that this lack of improvement could be due to suboptimal hyperparameter settings. We will continue to explore Focal Loss and other related methods to make the optimization more effective. Thank you again for your valuable suggestion!

We are honored and grateful to receive your positive evaluation. We sincerely appreciate your thorough review and the recognition of several strengths in our work, including the challenging new task, the valuable new dataset, and our essential network for USCOD. Hope we can address your concerns with the responses below.

---

Q1: More details about the CS12K are expected. For example, when manually annotating, how is the object judged to be as salient or camouflaged?

How do you judge whether an object is salient or camouflaged? Does this process align with popular datasets?

Ans for Q1:

For Scene A and B, we randomly selected 3,000 images from existing public datasets. For Scene D, we collected 1,564 images from COD10K and manually collected 1,436 images from the internet. For Scene C, we manually filtered data with coexisting salient and camouflaged objects from existing public datasets, resulting in a total of 342 images, including 210 from COD10K, 40 from CAMO, 92 from NC4K, 32 from AWA2, and 9 from LUSI, while also collecting and filtering 2,617 images from the internet. The data sources are explained in detail in Section 3.1 of the manuscript. For Scene A and B, we retained their original annotations, while Scene D did not require additional annotation. Therefore, we focus here on detailing the annotation process for Scene C.

• Initial Determination of Object Attributes: We invited 7 observers to perform the initial identification of salient and camouflaged objects in the images. A voting process was used to determine the salient and camouflaged objects in each image, with objects and their attributes receiving more than half of the votes being retained. We then used Photoshop to apply red boxes for salient objects and green boxes for camouflaged objects, which served as the reference for the subsequent mask annotation step.

• Mask Annotation: We invited 9 volunteers to perform detailed mask annotation for the dataset using the ISAT interactive annotation tool [1], which supports SAM semi-automatic labeling.

• Annotation Quality Control: After annotation, we invited an additional 3 observers to review and refine the results. Masks with imprecise or incorrect annotations were manually corrected.

In summary, we adopted a voting and feedback mechanism to ensure the quality of annotations, which aligns closely with the practices used in existing datasets [2-5]. Thank you for your suggestion. We will include more detailed information about dataset annotation in the Appendix of the revision.

---

---

Reference:

[1] https://github.com/yatengLG/ISAT_with_segment_anything

[2] Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan. Learning to detect salient objects with image-level supervision. In CVPR, 2017a.

[3] Guanbin Li and Yizhou Yu. Visual saliency based on multiscale deep features. In CVPR, 2015.

[4] Qiong Yan, Li Xu, Jianping Shi, and Jiaya Jia. Hierarchical saliency detection. In CVPR, 2013.

[5] Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and Ling Shao. Camouflaged object detection. In CVPR, pp. 2777–2787, 2020.

Thank you for your valuable time and thoughtful feedback. We greatly appreciate your recognition of our dataset, new metric, and the readability of our manuscript. Below are our responses to the concerns you raised, and we hope that these answers effectively address your questions.

---

Q1: Is the comparison with other models fair? Were the other models compared using the same training data as this model?

Ans for Q1:

We describe the training set and testing set used for the comparison experiments in L402-403 of the manuscript. Before evaluating the performance of each model, we retrain all the models using the same training set (CS12K training set with 8,400 images) as our USCNet to ensure fairness in the evaluation.

• After training, we evaluate all 16 models using the CS12K testing set, which consists of 3,600 images covering all scenes. We use mIoU, mAcc, and CSCS as evaluation metrics. The experimental results are shown in Table 2.

• Additionally, to further validate the effectiveness and robustness of our model regarding generalizability, we conduct tests on popular SOD datasets (DUTS, HKU-IS, and DUT-OMRON) and COD datasets (CAMO, COD10K, and NC4K), with all models uniformly trained using our CS12K training set. In these experiments, we adopt five metrics that are widely used in COD and SOD tasks: maximum F-measure ($F_β^{max}$ ↑), weighted F-measure ($F_β^{ω}$ ↑), Mean Absolute Error (MAE, M ↓), Structural measure (S-measure, $S_α$ ↑), and mean Enhanced alignment measure (E-measure, $E_ϕ^m$ ↑). This part of the experiment is described in Appendix §B of the manuscript.

---

Q2: The model design in this paper does not look very novel. Specifically, this paper proposes an adapter prompting strategy based on a cross-attention mechanism, which is very similar to [1].

Ans for Q2:

We have carefully reviewed the article [1] you provided and conducted a study on VRP-SAM. While both our model and VRP-SAM generate adapter prompts for SAM, the processes and objectives behind prompt generation in our approach differ significantly from those in VRP-SAM.

1. Differences in the Prompt Generation Process

• In VRP-SAM, a set of learnable queries $Q ∈ R^{N×C}$ is introduced, and two rounds of cross-attention and self-attention are performed. In the first round, $Q$ interacts with the visual features of the reference image to obtain $Q'_r ∈ R^{N×C}$. In the second round, $Q'_r$ interacts with the target image to generate $Q'_t ∈ R^{N×C}$, which ultimately serve as the visual reference prompt embeddings for SAM.

• Compared to VRP-SAM, to obtain discriminative Saliency, Camouflage and Background prompts for the USCOD task, our model employs three sets of static learnable queries {$Q_{S-S} ∈ R^{N×C}, Q_{S-C} ∈ R^{N×C}, Q_{S-B} ∈ R^{N×C}$} and three sets of dynamic queries {$Q_{D-S} ∈ R^{N×C}, Q_{D-C} ∈ R^{N×C}, Q_{D-B} ∈ R^{N×C}$} generated from image feature. Static queries focus on capturing general attribute features that remain consistent across samples, regardless of the input during inference. Unlike the static queries, dynamic queries are derived from the input image, enabling them to capture unique attribute features specific to the given sample. By integrating static and dynamic prompt queries, we construct three attribute prompt queries: {$Q_S ∈ R^{N×C}, Q_C ∈ R^{N×C}, Q_B ∈ R^{N×C}$}. These queries are concatenated into a unified prompt query $Q ∈ R^{3×N×C}$. Then, the combined query leverages self-attention to capture differences among the three attribute prompts, followed by cross-attention to integrate input image features and enhance their learning.

2. Differences in Optimization Objectives

• The Prompt Generator in VRP-SAM is designed to learn the visual features of the reference image, generating a general, class-specific visual semantic prompt.

• In comparison, our APG module extracts attribute features from the input image, generating three distinct, class-agnostic attribute prompts.

Thank you for providing a relevant work. We will include a discussion of VRP-SAM in the related work of the revision.

Reference:

[1] Sun, Yanpeng, et al. "VRP-SAM: SAM with visual reference prompt." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Q3: The authors repeatedly state that distinguishing salient and camouflaged objects within the same image is challenging. However, theoretically, as salient and camouflaged are antonyms, it should not be excessively difficult to differentiate between them.

Ans for Q3:

Although saliency and camouflage are conceptually opposites, distinguishing between salient and camouflaged objects is not straightforward in complex real-world scenarios. When both of them coexist within the same scene, they may have similar visual features or highly overlapping regions, increasing the difficulty of differentiating between them.

• When salient and camouflaged objects have similar visual features, the model cannot rely on a single characteristic to make the distinction. Instead, it must consider multiple visual cues within the scene. For example, in Fig. 8 (Row 1, Column 3) of the manuscript, two leopards appear together. Although their shapes are similar, they belong to different attributes—one is salient, and the other is camouflaged. In this case, the model must analyze local texture variations, surrounding differences, lighting conditions, and other multi-dimensional features to accurately differentiate between the two objects.

• When salient and camouflaged objects overlap or are in close proximity, the salient object may distract the model and interfere with its ability to perceive the camouflaged object. For example, in Fig. 8 (Row 5, Column 4) of the manuscript, a yellow spider is hidden within a yellow flower. The flower is the salient object, while the spider is camouflaged. The spider's camouflaged nature arises because it overlaps with the flower. In this case, the model needs to deeply analyze the contextual relationship between the two objects in order to accurately distinguish them.

This challenge is also reflected in our experimental results. As shown in Table 2 of the manuscript, the $IoU_S$ for salient objects in Scene C is, on average, 6.25% lower compared to Scene A, and the $IoU_C$ for camouflaged objects in Scene C is, on average, 25% lower compared to Scene B. These results highlight the challenge of distinguishing between salient and camouflaged objects in scenes that contain both.

Therefore, although these two terms are opposites, their distinction becomes blurred in complex, unstructured scenes, significantly increasing the difficulty of differentiating between them. To effectively address this challenge, the model must possess sufficient flexibility to learn and differentiate these objects in unconstrained scenes.

Dear Reviewer FwYu,

The deadline for posting messages to authors is approaching, and we may soon be unable to view your new comments. If you have any further concerns, please feel free to let us know, and we will do our best to address them before the deadline for posting messages to reviewers.

Thank you again for your valuable time and feedback.

Best regards,

Authors of #38

Q4: If VSCode and Spider do not need to be retrained, I am curious about their evaluation protocols, as well as that of EVP (Liu 2023). How are the prompts set for scenes A, B, C, and D? I cannot find any details regarding this, but I believe it is critical since it directly relates to the fairness and necessity of using the same type of prompt for the USCOD task. Thus, I hope the authors can make the evaluation protocols clearer (this is important).

Ans for Q4:

In order to ensure fairness in the evaluation, we retrain all models to be evaluated on the training set of CS12K (8,400 images), as detailed in L402-L403 of the manuscript.

• For the unified models, VSCode, Spider, and EVP, which require task-specific prompts for each dataset, we create two copies of the CS12K training set. One copy is used for SOD, with the ground truth being the SOD-only mask, and is used to train the prompts corresponding to the SOD task. The other copy is used for COD, with the ground truth being the COD-only mask, and is used to train the prompts corresponding to the COD task.

• VSCode and Spider are trained once using all 16,800 images (two copies of 8,400 images), while EVP is trained twice on the two separate training sets (each containing 8,400 images) to obtain the two task-specific prompts.

• During evaluation, all unified models perform inference on the testing set of CS12K twice, with the corresponding prompt enabled for each task. The first inference run generates the SOD results, and the second inference run generates the COD results. The final prediction is obtained by merging the SOD and COD predictions. For overlapping pixels, the attribute with the higher prediction value between the two tasks is chosen as the final attribute for that pixel.

Thank you very much for your careful review. We apologize for not providing a clear description of the evaluation details, and we will provide a more detailed explanation in the revision.

---

Q5: Regarding the CSCS metrics, based on the formulation, it seems they only consider the confusion between salient and camouflaged pixels, while the false positives and false negatives between salient/camouflaged pixels and the background are not considered. Therefore, we still need to look at mIOU to understand the quality of the method. However, mIOU can also reveal aspects such as the confusion between salient and camouflaged pixels. Thus, I wonder why we still need CSCS metrics?

Ans for Q5:

The CSCS metric is specifically designed to assess the degree of confusion between saliency and camouflage, which is a key aspect of model evaluation. While mIoU is indeed a valuable metric for evaluating a method's ability to distinguish saliency, camouflage, and background, CSCS provides a more focused evaluation of how well the model distinguishes between salient and camouflaged objects.

• In other words, CSCS allows us to directly measure the accuracy of distinguishing between these two attributes of objects. mIoU, while useful, does not fully capture this aspect, as it also accounts for the background class and may dilute the specific performance in distinguishing between salient and camouflaged objects.

• In many cases, it is necessary to combine both metrics for a more detailed evaluation of model performance. For example, in Table 2 of the manuscript, PFNet and ZoomNet have similar mIoU scores, but there is a noticeable difference in their CSCS values.

Therefore, both CSCS and mIoU are complementary metrics, with CSCS offering a more direct measure of performance on the salient/camouflaged distinction, and mIoU providing a broader view of overall segmentation quality.

---

Q6: If the paper is accepted, will the dataset be released in its “full version”? By this, I mean the version containing the entire training set (8,400 images and the corresponding masks) and the testing set (3,600 images and the corresponding masks), along with the scene types.

Ans for Q6:

Sure, if the paper is accepted, we guarantee that the complete dataset will be released for related domain research, including the entire training set (8,400 images and corresponding masks), the test set (3,600 images and corresponding masks), along with the scene types. We are committed to advancing research in the related field, and it would be an honor to contribute to its development.

We sincerely thank you for your time and insightful comments. We also appreciate your recognition of our dataset and the sufficiency of our experiments. Below are our responses to your concerns, and we hope that these answers address them effectively.

---

Q1: The motivation of this work is unclear. I am curious that why we need to jointly detect salient and camouflaged objects without any pre-defined prompts? I cannot see critical problems if using the pre-defined prompts. May the authors add more explanations regarding this?

Ans for Q1:

The motivation of this work is to solve the problem of jointly detecting salient and camouflaged objects in more complex and unconstrained scenes.

• Previous methods that required pre-defined prompts unify and integrate two independent binary classification tasks (SOD and COD). These methods work effectively in simplified scenes where only salient or camouflaged objects are present. However, in more complex scenarios where salient and camouflaged objects coexist within the same scene, they face limitations. Specifically, to jointly detect objects in such scenarios, these methods require two separate prompts for detecting each attribute, performing two binary classifications, and generating two binary masks. When merging the masks to obtain the final prediction, the overlap of pixels between the two masks can introduce uncertainty into the prediction result.

• In contrast, by jointly detecting both types of objects without pre-defined prompts, our model can learn to differentiate these objects more flexibly and robustly. To be specifically, the final predictions are generated internally by the model, eliminating the need for post-processing. This approach enables the model to learn the attributes of uncertain pixels during training, effectively addressing the issue of pixel overlap. In this way, our model is able to make more accurate predictions in more realistic and complex scenarios.

---

Q2: The novelty of this work is very limited. I cannot see the technique novelty. One APG is proposed, but the APG design is not novel, since using different type of prompts for different task setting is usual. The idea of the proposed method is not novel as well. May the author highlight it?

Ans for Q2:

We understand your concern regarding the perceived novelty of our method. Using different types of prompts for different tasks is indeed a common approach. However, in these methods, the prompts are independent and merely serve to distinguish between tasks, with no interaction or learning between them. This can result in the model being unable to learn sufficiently discriminative prompts for the USCOD task.

• In contrast, our APG module focuses on generating distinct and discriminative prompts for the three attributes (saliency, camouflage, and background) within the same scene. It models the interrelationships between the prompts of three attributes, thereby generating discriminative attribute prompts that better differentiate the salient and camouflaged patterns of objects.

• Additionally, our architecture decouples attribute distinction from mask reconstruction by freezing the SAM mask decoder, which allows the network to focus on learning the attributes in a more targeted manner.

This combination of modeling attribute relationships and decoupling classification from mask reconstruction is what sets our approach apart from existing techniques and contributes to its novelty.

---

Q3: In L088-L091, "The visual prompt of EVP, as part of the model, needs to be retrained according to the datasets of different tasks and requires pre-defining the category of the detection task. Similarly, the task prompts in VSCode and Spider also need to be pre-defined. Similarly, the task prompts in VSCode and Spider also need to be pre-defined." The "Similarly" here is confusing. Do VSCode and Spider need to be retrained? Or only require pre-defined prompts? Thanks for classify this!

Ans for Q3:

To clarify, the "retrain" in EVP refers to the need to train different visual prompts multiple times based on the datasets of different tasks, resulting in task-specific visual prompts. In contrast, for VSCode and Spider, during training, it is only necessary to distinguish between the prompts across datasets of different tasks, and combine multiple datasets of different tasks for training once. The term "Similarly" in this context refers to the fact that "task prompts need to be pre-defined" during inference, rather than retraining the model. Sorry for the confusion and we will revise the wording in the revision accordingly.

Dear Reviewer 224k,

We apologize for not being able to respond to all reviewers' comments simultaneously. We wanted to respond to each reviewer’s comments one by one to ensure the comprehensiveness of our responses.

In the past few days, we have made every effort to carefully address each reviewer’s concerns, providing explanations and conducting additional experiments. Since Reviewer Htmz's comments included some additional experiments, we only responded after confirming that we had completed all the necessary experiments.

We have now provided our first round of responses to all reviewers’ comments, including all of Reviewer Htmz's comments. We kindly ask you to review our responses again, and we are committed to addressing any other issues or concerns you may have.

Thank you once again for your valuable feedback.

Best regards,

Authors of #38

Q5: It is unclear how the static prompt queries (SPQ) are generated. Moreover, the operation to predict the coarse operation is also poorly explained. It is hard to see the role of combining dynamic prompt queries (DPQ) and static prompt queries. Moreover, the motivation for the DPQ method is also unclear; authors should explain the reason why a learning component is required for DPQ calculation. Additionally, authors should explain the motivation and how they employ self-attention with salient, camouflaged, and background queries and further generate the prompts for the same.

Ans for Q5:

Our APG module is designed to extract the discriminative attribute prompts of objects. Our insight is that the detection of salient and camouflaged objects within a sample requires consideration of features across two dimensions:

• Sample-generic features: For all samples, characteristics such as the size, position, color, and texture of the object serve as important generic features for distinguishing salient and camouflaged objects. These features are applicable in most scenarios and can act as universal criteria for judgment.

• Sample-specific features: Relying solely on sample-generic features may not suffice in certain complex situations. For instance, when the salient and camouflaged objects share similar colors or categories, sample-generic features alone are insufficient for effective differentiation. In such cases, it is crucial to consider the specific contextual information within the sample and learn features that are closely associated with the current sample to assist in making an accurate judgment.

By combining these two types of features, we generate more discriminative attribute prompts, enabling more accurate differentiation between salient and camouflaged objects.

Based on this, we incorporate two types of queries in the design of the APG module, each responsible for capturing features at different dimensions:

• Static Prompt Query (SPQ): Extracts the sample-generic features, capturing attribute information applicable to all samples.

• Dynamic Prompt Query (DPQ): Extracts sample-specific features, focusing on the unique contextual information of the current sample.

The process for obtaining SPQ, DPQ, and generating the attribute prompts for saliency, camouflage, and background is as follows:

1. SPQ: We randomly initialize three sets of learnable queries, namely the static saliency, camouflage, and background prompts queries, denoted as {$Q_{S-S} ∈ R^{N×C}, Q_{S-C} ∈ R^{N×C}, Q_{S-B} ∈ R^{N×C}$}, where $N$ represents the number of queries, and $C$ represents the dimensionality of the queries (kept consistent with the dimensionality of the original SAM prompts). These static prompt queries remain unchanged during inference, regardless of the input image, and are designed to capture generic features applicable across all samples.

2. DPQ: We first utilize the obtained coarse prediction as an attention map to focus on the relevant regions of saliency, camouflage, and background in the image, isolating attribute-specific features from the global image features. Then, a learnable component (refer to the Linear in Fig. 4) is used to downsample the extracted features, aligning them with the dimension of the SPQ. The aligned queries are then treated as DPQ, i.e., the three sets of dynamic prompt queries, representing the dynamic saliency, camouflage, and background prompts queries {$Q_{D-S} ∈ R^{N×C}, Q_{D-C} ∈ R^{N×C}, Q_{D-B} ∈ R^{N×C}$}. These dynamic prompt queries change during inference based on the input image and are used to extract sample-specific features.

3. Attribute Prompt: Combine SPQ and DPQ through a summation operation, resulting in the combined queries {$Q_S ∈ R^{N×C}, Q_C ∈ R^{N×C}, Q_B ∈ R^{N×C}$}. These combined queries are then concatenated to form the attribute prompt query $Q ∈ R^{3×N×C}$. A self-attention is employed to explore the similarities and differences between the queries, learning the relationships among saliency, camouflage, and background features, producing $Q' ∈ R^{3×N×C}$. Subsequently, cross-attention is applied to further enhance the learning among the three attribute queries based on the image features, resulting in $Q'' ∈ R^{3×N×C}$. Finally, the $Q'' ∈ R^{3×N×C}$ serves as the Attribute Prompt, which is separated into the saliency, camouflage, and background prompts {$P_S ∈ R^{N×C}, P_C ∈ R^{N×C}, P_B ∈ R^{N×C}$}.

Thank you for your suggestion. We will provide a more detailed explanation of the working mechanism of APG in the relevant section of the revision.

Thank you for your careful reading and the valuable questions you raised. We also appreciate your recognition of our dataset, new metrics, and the performance of our model. In the past few days, we have conducted some experiments on the latest methods and provided detailed explanations addressing your concerns. Below, we provide our detailed responses to your comments and hope that these clarifications address your concerns effectively.

---

Q1: This work tackles only extremes, salient or camouflaged objects. What about intermediate scenarios, for e.g., non-salient or less-salient objects. What about objects with complex backgrounds? If we are trying to generalize, it should cover the entire range, not just the two extremes.

Ans for Q1:

In classic SOD and COD tasks, such as [1] [2], objects that are neither salient nor camouflaged—i.e., non-salient or less-salient objects, and non-camouflaged or less-camouflaged objects—are typically treated as background. These objects either lack distinct visual features or fail to exhibit strong visual similarity to their surroundings. We maintain this setting, focusing on distinguishing the salient and camouflaged patterns of objects, while ignoring generic objects.

---

Q2: The scenario of multiple objects seems missing from the dialogue. Although the authors have proposed the dataset by collecting most of the images from previous benchmark datasets, the presence of multiple camouflaged/salient objects (as per real-world scenario) seems to be missing from the dataset or model itself.

Ans for Q2:

Our CS12K dataset contains images with different numbers of objects. To show it more clearly, we have counted the distribution of images with different numbers of objects in CS12K, as shown in the following Table R1. Indeed, several previous popular SOD datasets (DUTS [1], HKU-IS [3], and DUT-OMRON [4]) and COD datasets (CAMO [5], COD10K [2], and NC4K [6]) also contain scenes with multiple camouflaged and/or salient objects. Thank you for your suggestions and we will include an analysis of this in the revision.

Table R1: Distribution of Images with Different Numbers of Objects in CS12K

---

---

Reference:

[1]Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan. Learning to detect salient objects with image-level supervision. In CVPR, 2017a.

[2]Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and Ling Shao. Camouflaged object detection. In CVPR, pp. 2777–2787, 2020.

[3]Guanbin Li and Yizhou Yu. Visual saliency based on multiscale deep features. In CVPR, 2015.

[4]Chuan Yang, Lihe Zhang, Huchuan Lu, Xiang Ruan, and Ming-Hsuan Yang. Saliency detection via graph-based manifold ranking. In CVPR, 2013.

[5]Trung-Nghia Le, Tam V Nguyen, Zhongliang Nie, Minh-Triet Tran, and Akihiro Sugimoto. Anabranch network for camouflaged object segmentation. CVIU, 184:45–56, 2019.

[6]Yunqiu Lv, Jing Zhang, Yuchao Dai, Aixuan Li, Bowen Liu, Nick Barnes, and Deng-Ping Fan. Simultaneously localize, segment and rank the camouflaged objects. In CVPR, pp. 11591–11601, 2021.

Q3: Salient and camouflaged objects belong to different distributions based on their visual appearance and structural complexity. However, the loss function or the network does not cater to that. Is there a way to analyze this difference in the distribution of salient and camouflaged objects so that it does not impact the final prediction?

Ans for Q3:

As you mentioned, the salient and camouflaged objects belong to different distributions, which is a critical issue. Our network does not independently learn salient and camouflaged objects but instead models the distributional differences between salient and camouflaged patterns internally. Specifically, the APG module models the relationships and distinctions among the three attributes (saliency, camouflage, and background), generating three discriminative attribute prompts: $P_S$, $P_C$, and $P_B$ (refer to Fig. 4). This reduces the impact of distributional differences between salient and camouflaged objects on the final prediction. The specific mechanism of APG is detailed in Ans for Q5. Thanks to this, when both salient and camouflaged objects are present in a scene, the network can accurately distinguish and identify them, which is the primary problem we aim to solve.

Table R2 presents the experimental results of our model with and without the APG module in Scene C (Salient and camouflaged objects coexist) and overall scenes. This results show that our APG module can simultaneously improve the model's performance in detecting both salient and camouflaged objects across all scenes, with a particular improvement in Scene C. Compared to the results without APG, the model with APG achieves improvements of 5.79 in $IoU_S↑$ and 7.53 in $IoU_C↑$ in Scene C, and improvements of 4.15 in $IoU_S↑$, 4.63 in $IoU_C↑$, 4.63 in $mIoU↑$, 4.63 in $mAcc↑$, and 1.63 in $CSCS↓$ in Overall Scenes. This indicates that our approach is better adapted to the distributions of salient and camouflaged objects across various scenarios, particularly in scenes where both salient and camouflaged objects coexist.

Table R2: Effectiveness of APG in Scene C and Overall Scenes

To further validate the ability of the APG module in promoting the network's learning of discriminative salient and camouflaged features, we used Euclidean Distance to evaluate the feature distribution differences between salient and camouflaged features on the entire CS12K testing set, both with and without using APG. The formula for calculating the Euclidean Distance is as follows:

$Dist = || Fea_s^{(i)} - Fea_c^{(i)} ||^2 = Σ_j ( Fea_s^{(i)}[j] - Fea_c^{(i)}[j] )^2$,

where $Fea_s^{(i)}$ and $Fea_c^{(i)}$ represent the saliency features and camouflage features of the $i$-th sample, respectively, and $j$ is the feature dimension. The $Fea_s$ and $Fea_c$ come from the saliency token and camouflage token generated by the SAM decoder, respectively, where $j$ is the token dimension of 256. We evaluated this on a total of 3,600 images from the CS12K testing set, and the results are as follows:

• Without APG, the average Euclidean Distance between salient and camouflaged features is 306.1.

• With APG, the average Euclidean Distance between salient and camouflaged features is 391.7.

It can be observed that when using APG, the Euclidean Distance between saliency and camouflage features in the feature space increases significantly, indicating a larger spatial separation between them. This suggests that our method effectively captures the distributional differences between salient and camouflaged objects.

Q6: In line 355, the authors state that they employ self-attention, while in line 352, they mention cross-attention. This statement is contradictory and unclear.

Ans for Q6:

Are you referring to "cross-attention" in line 362 and "self-attention" in line 355? There should be a comma after "queries" in line 355. What we want to express in line 355 is "we employ self-attention to establish relationships between queries, and query-to-image (Q2I) attention to interact with image embedding." Thank you for your careful reading. We apologize for any confusion caused by our phrasing, and we will revise this sentence in the revision.

---

Q7: In table 5, there seems to be some ambiguity in the results. Specially for ICEG, the results in the manuscript provides different results than what the paper has provided (in all combined settings). Some light on this issue is required.

Ans for Q7:

Table 5 shows the generalization performance of all methods on the testing sets of publicly available COD datasets. Before testing, all models are trained on our CS12K training set, which differs from the training set used in the original ICEG paper. Therefore, the results in the manuscript differ from those in the original paper. The training data is discussed in L860 of the manuscript.

Additionally, the ICEG backbone in Table 5 of the manuscript is ResNet50, which is the common setting in the original paper. We also trained ICEG with other backbones from the original paper on our dataset, evaluated it on the CS12K testing set, and conducted generalization experiments on popular SOD and COD datasets. The results of the comparison experiments are shown in Table R5, and the results of the generalization experiments are shown in Table R6 and Table R7.

Table R5: Performance of ICEG with diverse backbones on the CS12K testing set.

Table R6: Performance of ICEG with diverse backbones on the DUTS, HKU-IS, and DUT-OMRON testing sets. DUT-O denotes DUT-OMRON.

Table R7: Performance of ICEG with diverse backbones on the CAMO, COD10K, and NC4K testing sets.

Q4: Although the authors leverage the SAM-adapter idea, it's strange that they haven't compared with it, which does provide COD results. Also, please provide results with other recent COD and SOD works. Some of them are listed below:

CamoFocus [1]

CamoDiffusion [2]

CamoFormer [3]

PGT [4]

Ans for Q4:

In Table 2 of the manuscript, the last row for the COD models shows SAM2-Adapter [6], which is the 2.0 version of SAM-Adapter [5] and performs better than SAM-Adapter. To test the upper limit of SAM-Adapter, we evaluated SAM2-Adapter. The experimental results for SAM-Adapter and SAM2-Adapter are shown in Table R3. Thank you for the reminder and we will include the SAM-Adapter results in the revision of Table 2.

Table R3: Quantitative comparisons with SAM-Adapter and SAM2-Adapter.

Regarding the four latest COD works you mentioned, except for CamoFocus [1], which has not yet released its code, we have conducted experiments on the other three COD methods for the USCOD task. After training the three methods on the CS12K training set, we evaluate them on the CS12K testing set, and the experimental results are shown in Table R4. We can see that our proposed USCNet still performs best in all evaluation metrics. Thank you for providing these new methods and we will include these experimental results in the version of Table 2.

Table R4: Quantitative comparisons with CamoDiffusion, CamoFormer and PGT.

---

---

Reference:

[1] Abbas Khan, Mustaqeem Khan, Wail Gueaieb, Abdulmotaleb El-Saddik, Giulia De Masi, and Fakhri Karray. Camofocus: Enhancing camouflage ob- ject detection with split-feature focal modulation and context refinement. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2024, Waikoloa, HI, USA, January 3-8, 2024, pages 1423–1432. IEEE, 2024.

[2] Zhongxi Chen, Ke Sun, and Xianming Lin. Camodiffusion: Camouflaged object detection via conditional diffusion models. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Eighth AAAI Confer- ence on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innova- tive Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 1272–1280. AAAI Press, 2024.

[3] Bowen Yin, Xuying Zhang, Deng-Ping Fan, Shaohui Jiao, Ming-Ming Cheng, Luc Van Gool, and Qibin Hou. Camoformer: Masked separable atten- tion for camouflaged object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–14, 2024.

[4] Rui Wang, Caijuan Shi, Changyu Duan, Weixiang Gao, Hongli Zhu, Yunchao Wei, and Meiqin Liu. Camouflaged object segmentation with prior via two-stage training. Comput. Vis. Image Underst., 246:104061, 2024

[5]Tianrun Chen, Lanyun Zhu, Chaotao Deng, Runlong Cao, Yan Wang, Shangzhan Zhang, Zejian Li, Lingyun Sun, Ying Zang, and Papa Mao. Sam-adapter: Adapting segment anything in underperformed scenes. In ICCV, pp. 3367–3375, 2023.

[6]Tianrun Chen, Ankang Lu, Lanyun Zhu, Chaotao Ding, Chunan Yu, Deyi Ji, Zejian Li, Lingyun Sun, Papa Mao, and Ying Zang. Sam2-adapter: Evaluating & adapting segment anything 2 in downstream tasks: Camouflage, shadow, medical image segmentation, and more. arXiv preprint arXiv:2408.04579, 2024.

Thank you for your feedback. We hope our response can address your concerns.

---

Although salient and camouflaged objects are in opposing directions, establishing a connection between them during training is still necessary. In fact, both VSCode and our method establish a connection between prompts, but the way the connection is formed differs due to the different focuses of the models. Specifically, VSCode uses two separate prompts to handle the SOD and COD tasks separately, but these prompts are not completely independent in their learning. It uses discriminative loss to reduce the similarity between different prompts, thereby indirectly establishing a connection to learn the feature differences between salient and camouflaged objects across different samples (inter-image samples), effectively handling seven different multimodal SOD and COD tasks. In contrast, our joint prompt, designed for USCOD, directly models the interaction between prompts within the network through the APG module. This approach is more suited for learning the feature distribution of salient and camouflaged objects within samples (intra-image samples).

In the USCOD task, the joint prompt has two advantages:

• Adaptation to Complex Scenes: Salient and camouflaged objects are not merely opposites in terms of similarity, especially when they coexist in the same scene. For example, camouflaged objects may be hidden within salient objects (e.g., flowers and insects of the same color) or may belong to the same species with similar morphological features, where factors like lighting or background cause one to appear salient and the other camouflaged. In such complex scenes, salient and camouflaged objects may share certain features, and simply reducing prompt similarity may not suffice to capture the intricate relationship between them. Our APG module directly models the interaction between prompts, enabling the model to flexibly adapt to the specific feature distribution of the scene, rather than being limited to simple similarity measures. This approach allows for better handling of the complex and diverse scenarios in the USCOD task.

• Avoiding Pixel Uncertainty: VSCode has demonstrated strong generalization across seven different multimodal SOD and COD tasks, but it primarily targets scenes with only single attribute objects (Scene A and B in the manuscript). When aiming to detect salient and camouflaged objects in Scene C (salient and camouflaged objects coexist), VSCode needs to separately use the SOD and COD prompts to handle the same image, resulting in two independent masks. This approach often leads to overlapping pixel regions in the detection results. For example, in Figure 4 of the VSCode paper, when using the SOD prompt, the network detects the salient palace and part of the camouflaged person, while using the COD prompt, the camouflaged person is detected again, resulting in overlapping pixels and increased detection uncertainty. In contrast, our joint prompts directly model the relationship between salient and camouflaged objects through the APG module, automatically learning uncertain pixels within the same scene without the need for post-processing. This approach not only alleviates the overlapping pixel issue but also enhances the stability and accuracy of the detection results.

Overall, VSCode demonstrates strong generalization across seven different multimodal SOD and COD tasks, effectively learning the feature differences between salient and camouflaged objects across different samples (inter-image samples) by indirectly establishing connections between prompts. In contrast, our model focuses on the USCOD task and uses joint prompts to directly model the interaction between prompts, enabling mutual learning between the two attributes during training. This direct interaction approach is better adapted to the feature distribution of salient and camouflaged objects within the same sample (intra-image sample), effectively improving the model's performance on the USCOD task.

---

If you have any further questions or need additional clarification, please feel free to let us know, and we will respond promptly.

Dear Reviewer Htmz,

We have provided responses and additional experimental results for your concerns, and we hope that we have addressed most of your questions. Since this is toward the end of the rebuttal, please don't hesitate to let us know if you have any final questions or comments. Thank you!

Best regards,

Authors of #38

Thank you for your feedback. We hope our response can address your concerns.

---

We are not dismissing prior research on foreground extraction. This paper does not aim to overturn the task setting of SOD and COD but rather extends them. Previous SOD and COD methods assume that a salient object and a camouflaged object will not appear in an image simultaneously, and learning saliency and camouflage inter-image samples (scene A and B in the paper), while our work introduces a more relaxed setting: enabling the model to simultaneously learn saliency and camouflage intra-image sample (scene C in the paper), thus making the model more unconstrained.

The general foreground extraction you mentioned is typically more generalized, aiming only to separate foreground from background without distinguishing the specific attributes of objects within the foreground. The key difference in our approach is that we shift the region of interest from the entire foreground to salient and camouflaged objects, explicitly assigning these attributes.

In practical applications, the two tasks have different focus areas:

• General foreground extraction is applied to the foreground detection needs in generic scenarios, such as image editing, video surveillance, and video monitoring, without considering the saliency or camouflage of the objects.

• The USCOD task is more focused on special scenario analysis, where it plays a crucial role in improving operational efficiency (such as post-disaster rescue, multi-level lesion detection) and enhancing safety (such as diving hazard warnings, military surveillance). In these scenarios, salient objects are often considered a higher priority, prompting quicker decision-making or action, while camouflaged objects are seen as potential risks, prompting targeted actions to address them. Simultaneously detecting both enhances decision-making comprehensiveness and effectiveness.

The typical real-world applications of USCOD are as follows:

• Military Surveillance and Enemy Reconnaissance: In a military environment, salient objects could be large military equipment (such as tanks, helicopters), which are usually clear sources of threat and require immediate attention and action. Camouflaged objects could be soldiers or equipment hidden in vegetation or camouflage materials, often overlooked but potentially carrying deadly threats. Salient objects require quick identification and military strikes to weaken enemy combat capabilities, while camouflaged objects, as hidden threats, need extra attention to prevent oversight. Simultaneously detecting both can balance addressing direct threats and eliminating potential ones, thereby enhancing battlefield situational awareness.

• Post-Disaster Search and Rescue: After a disaster, salient objects could be clear signs of life in the rubble (such as visibly trapped individuals), representing the immediate focus for rescue efforts, where quick identification and action can save lives. Camouflaged objects might be life signs concealed due to obstruction or chaotic environments (such as partially buried survivors), often overlooked and more difficult to rescue. Prioritizing the rescue of salient objects increases efficiency, while camouflaged objects might require more manpower and resources to reach. Simultaneously detecting both helps allocate resources effectively and ensure accurate rescue orders.

• Multi-Level Lesion Detection: In medical imaging, salient objects might include obvious tumors or organ damage, typically the first focus for doctors. Camouflaged objects could be lesions with blurred boundaries or early-stage abnormalities, which may indicate early-stage diseases or more complex issues. Detecting salient lesions allows doctors to quickly determine treatment priorities, while identifying camouflaged lesions helps catch potential health problems early, preventing disease progression. Simultaneously detecting both improves diagnostic efficiency and accuracy, helping to develop the best treatment plan for patients.

• Diving Hazard Warnings: During a dive, salient objects might include coral or schools of fish that attract divers' attention, contributing to an enriching visual experience. Camouflaged objects could be dangerous creatures like stonefish or moray eels, whose concealment increases potential risks, and failure to detect them could jeopardize the diver's safety. Detecting salient objects can guide divers to interesting spots, enhancing the diving experience, while identifying camouflaged objects helps divers avoid potential dangers and ensure safety. Simultaneously detecting both improves the diving experience and reduces safety risks.

These applications are also discussed in §G of the Appendix in the revision. If you have any further concerns, we will do our best to address them.

Dear Authors,

Thanks for the rebuttal.

However, my fundamental question on covering the range between salient and camouflaged still remains.
If you are saying less salient or less camouflaged objects should be treated as background, then all the research done over the years on foreground extraction in complex scenes appears meaningless.

My viewpoint is the following: If you are covering extremes, you should cover everything in-between as well, and call it generic foreground extraction instead. You should be aiming for foreground extraction regardless of whether it's salient, less salient, less camouflaged or camouflaged. I just don't understand what purpose does it serve by having a joint network for the two extremes. Can you think of any real-world application/setting for this?