CamoVid60K: A Large-Scale Video Dataset for Moving Camouflaged Animals Understanding

Q1 Legitimacy of Using Existing Dataset Masks and Dataset Contribution.

A1 As shown in Table 1, only CAD and MoCA-Mask datasets provided mask annotation, while MoCA dataset provided bounding box annotation. However, they only provided annotations for every 5 frames, resulting in only small amount of annotated frames (e.g. only 4,691 annotated frames of 22,939 frames from MoCA-Mask). In addition, the annotated mask from MoCA-Mask is image-based annotation, which is different from ours (json format as shown in Figure 3). Therefore, to ensure consistency and meet the dense (every frame) annotation requirements of our dataset, we re-annotated all videos sourced from existing datasets. Finnaly, in the Appendix A1, we mentioned that we collected 1,929 videos (including the videos from the existing datasets), and in Appendix A2, we manually reviewed and removed blurry or irrelevant videos (e.g. those with clearly visible animals), resulting in 218 videos with 62,774 frames.

Q2 Necessity of Dense Annotations.

A2 Previous studies from general video object segmentation [13-15] and special domains (camouflaged or salient objects) [16-19] have shown that the dense or frame-by-frame annotations benefit model learning by capturing temporal dynamics more effectively. In addition, [18,19] proved that utilizing pseudo-label (because they don't have GT labels for every frame) can improve the performance of video approaches as they can leverage temporal information to suppress the label noises.

[13] Zhang et al., Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions, CVPR 2013.

[14] Perazzi et al., A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation, CVPR 2016.

[15] Xu et al., YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark, ECCV 2018.

[16] Wang et al., Video Salient Object Detection via Fully Convolutional Networks, T-IP 2018.

[17] Chen et al., Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection, CVPR 2018.

[18] Yan et al., Semi-Supervised Video Salient Object Detection Using Pseudo-Labels, ICCV 2019.

[19] Cheng et al., Implicit Motion Handling for Video Camouflaged Object Detection, CVPR 2022.

Q3 Concerns about the use of Perceptual Camouflage Score.

A3

• Sorry for confusion and we thank the reviewer to point out the Appendix A2. Our data filtering process include both manually review by human observation and perceptual camouflage score. In most of the cases, our observation align with the perceptual camouflage score. We will enhance this description in the revised version.
• For the examples of Perceptual Camouflage Score in the supplementary, we used some same sets with Lamdouar’s original paper to show the illustration of low and high scores, and also verify the reported scores in Lamdouar’s original paper and our computed scores, which are mostly similar.

Q4 Pipeline Innovation and Contribution.

A4 Our main contribution is a dataset which includes data, annotations, simple baseline for future research and benchmark. In addition, please prefer to the answer of Q2 of Reviewer nU6R for our discussion of the pipeline.

Q1 Concerns about thoroughness in dataset labeling and research focus.

A1 We appreciate the reviewer’s concerns regarding the labeling process, and also the complement on an efficient pipeline, reducing both time and manual labor. We would like to clarify that while we rely on off-the-shelf models with human assistance (providing points as prompt for SAM model) to generate initial annotations, a rigorous manual refinement process is undertaken to ensure accuracy and relevance to camouflaged animal analysis. The refinement stage involves domain experts (the authors have been working on scene understanding field for more than a decade) validating and correcting the propagated masks (generated by XMem method) to mitigate errors and improve data quality.

Q2 Perceived lack of innovation in the baseline model.

A2 While the architecture components are sourced from existing models, our contribution lies in their specific integration and optimization tailored to camouflaged animal understanding (subtle motion cues and low contrast between the animal and its surroundings). It is based on Mask2Former architecture and act as a baseline for both object detection and motion segmentation tasks. The main difference is that we extend Mask2Former, which takes only one RGB image as input to take both sequence of RGB image and optical flow. Then, instead of directly outputting the prediction from Decoders, we output both the mask and bounding box prediction from the aggregation of the motion features (from Pixel Decoder) and the mask embedding (from Transformer Decoder).

Q3 Clarification on model initialization and training strategy.

A3 As mentioned in line 369, we adopt SINet-v2 architecture for our visual encoder. In detail, SINet-v2 utilized ResNet50 as backbone with weights pretrained on ImageNet dataset. We will explicitly include the ResNet50 backbone in the revised manuscript.

Q4 Exploration of distinctive challenges in camouflaged animal understanding.

A4 As mentioned in lines 50-53, our dataset potentially can help deep learning methods for monitoring and surveillance of wildlife animals, such as the camera trap method. Then, with the captured data, we can further study biodiversity. From the reference works, Troscianko et al. discusses how isolating camouflaged animals from their backgrounds allows researchers to gather precise data on their behavior, habitat preferences, and how their camouflage strategies evolve over time. Yunqiu et al. [2] highlights how modeling the conspicuousness of camouflaged objects enhances our understanding of camouflage and animal evolution.

Q5 Benchmarking methodology and inclusion of recent methods.

A5 Please prefer to our answer of Q2 of Reviewer JSBg above.

Q6 The paper mainly focuses on identifying salient objects in scenes. It would be interesting to explore how the model handles situations with multiple instances of camouflaged animals.

A6 We think that there is a misunderstanding. Our paper focuses on camouflaged objects (animals), which is essentially the opposite of salient objects. Regarding the multiple instances of camouflaged animals, our model's current architecture based on Mask2Former, inherently supports scalability to multi-instance detection. We hypothesize that minor modifications, such as adding a dedicated module for instance separation or leveraging ensemble approaches, would enhance its multi-instance segmentation performance [11,12].

[11] Cheng et al., "Mask2Former for Video Instance Segmentation," arXiv preprint 2112.10764, 2021.

[12] Lei et al., "Mask-Free Video Instance Segmentation," CVPR, 2023.

Question The paper focuses more on dataset development and benchmarking rather than presenting innovative new algorithms or techniques. Consequently, if the contribution to the field of VCOD is limited to proposing a dataset containing only 60k video frames, it may not be sufficient to consider the overall impact of the paper substantial. Moreover, I have noticed that the dataset proposed by the authors does not seem to facilitate advancements in other fields.

Answer

• Our paper aim to fill the gap of lacking a diverse, large-scale, densely and accurately annotated video dataset of camouflaged animals. That's why the primary area of our submission is the datasets and benchmarks.
• In addition, as shown in Table 1 and lines 87-100, our dataset surpasses all previous datasets in terms of the number of videos, frames, and species included. Unlike previous datasets that annotated every 5 frames, our dataset offers annotations for every single frame. Additionally, we provide a wider variety of annotation types (animal categories, bounding boxes, annotated masks, pseudo-label optical flow, referring expressions), making it a more effective benchmark for camouflaged animal video understanding tasks. Finally, our dataset supports a broad range of downstream tasks, including classification, detection, segmentation (semantic, referring, motion), and optical flow estimation.
• Lastly, in our Conclusion section, we have discussed the impact on animal studies (lines 513-518). In detail, Troscianko et al. [7] explore the impact of camouflage on predator-prey interactions. Norouzzadeh et al. [8] demonstrates the effectiveness of deep learning in wildlife identification and monitoring. Beery et al. [9] and Fanny et al. [10] discuss the use of bounding boxes and deep learning in wildlife imagery for ecological research and conservation efforts.

[7] Troscianko et al., "Quantifying camouflage: how to predict detectability from appearance." BMC Ecology and Evolution, 2017.

[8] Norouzzadeh et al., "Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning." PNAS, 2018.

[9] Beery et al., "Recognition in terra incognita." ECCV, 2018.

[10] Fanny et al., "DeepWILD: Wildlife Identification, Localisation and estimation on camera trap videos using Deep learning." Ecological Informatics, 2023.

Q1 The dataset is of relative small scale.

A1 As shown in Table 1 and lines 87-100, our dataset surpasses all previous datasets in terms of the number of videos, frames, and species included. Unlike previous datasets that annotated every 5 frames, our dataset offers annotations for every single frame. Additionally, we provide a wider variety of annotation types (animal categories, bounding boxes, annotated masks, pseudo-label optical flow, referring expressions), making it a more effective benchmark for camouflaged animal video understanding tasks. Our dataset supports a broad range of downstream tasks, including classification, detection, segmentation (semantic, referring, motion), and optical flow estimation.

Q2 The compared models are quite old, for example, MG was published in 2021, SLT-Net was published in 2022. In addition, as far as I know, MG was trained with self-supervised learning, thus the comparison is not that fair.

A2

• Regarding the age of the models: The selection of MG (2021) and SLT-Net (2022) was based on their relevance as established benchmarks in the field. These models represent widely acknowledged milestones for the specific tasks we address, and their inclusion provides context for evaluating the comparable performance introduced by our method. In addition, there are many recent published methods [1-5] and under review method [6], but to provide the most direct comparison, we focused on models with publicly available implementations (no available source code for [1,4-6]) and standardized evaluations (it is unfair to compare with SAM-based methods [2,3]).
• On fairness of the comparison with MG (self-supervised learning): we acknowledge the differences in training paradigms between MG (self-supervised learning) and our method (fully supervised). The inclusion of MG in our comparisons was not to claim a direct superiority but rather to provide a broader perspective on how our method performs relative to diverse approaches, including self-supervised ones. To ensure fairness, we will explicitly highlight these differences in the revised manuscript, emphasizing that MG’s performance reflects its specific training paradigm.

[1] Yu et al., Tokenmotion: Motion-Guided Vision Transformer for Video Camouflaged Object Detection VIA Learnable Token Selection, ICASSP 2024.

[2] Meeran et al., SAM-PM: Enhancing Video Camouflaged Object Detection using Spatio-Temporal Attention, CVPRW 2024.

[3] Hui et al., Endow SAM with Keen Eyes: Temporal-spatial Prompt Learning for Video Camouflaged Object Detection, CVPR 2024.

[4] Hui et al., Implicit-Explicit Motion Learning for Video Camouflaged Object Detection, T-MM 2024.

[5] Bideau et al., The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields, IJCV 2024.

[6] Zhang et al., Explicit Motion Handling and Interactive Prompting for Video Camouflaged Object Detection, arXiv 2024.

Q3 The contribution on the architecture design is minor, and trained on the video dataset with complete supervised learning, which limits the value of proposed method, as it is not scalable, as has been done in previous work, either with synthetic data, or self-supervised learning.

A3 Our proposed method is a simple pipeline which aims to provide a baseline to further explore our dataset on both motion detection and segmentation tasks of camouflaged animals.

Q4 Could the authors evaluate more new models ?

A4 Please prefer to our answer of Q2 above.