Towards Natural Image Matting in the Wild via Real-Scenario Prior

1. Novelty is not enough. Even though some tricks are used to construct a dataset, the matting model do not contain enough novel method.
2. The intro about matting dataset missed some existing datasets, training set and test set. For instance, RefMatte, ICM-57 and so on.
3. The ablation about the trimap loss is not clear. If you do not use the loss, what is used in your matting decoder to replace the trimap? In fact, I am concerned about the contribution of predicting trimap from SAM.

Despite the strong performance and valuable datasets, I still have some concerns.

1. Almost all of current matting datasets are manually labeled, but the authors just use trimap-based matting models to label it. So, whether the label can be called ground-truth is a question to be discussed. Is there any data selecting to verify the correctness of the dataset?

2. From the results, the update of SAM version can introduce the improvement, so I wonder the result if MatAny is equipped with SAM2. Also, the results of original SAM/HQ-SAM/SAM2 is recommended to illustrate at the table.

3. The authors provide the trainable parameters, but maybe the inference speed is also needed.

4. Insufficient ablation study of network design. It will reveal more understanding if the authors can conduct ablation study to show why the lora and matting adapter is needed.

1. The major contributions for this paper is the dataset. However, I have a very big concern on this part.

The way how authors generate the final 'high-quality' alpha matte is using a off-the-shelf method ."Finally,we forwardt he following trimapT(x) to the trained trimap-based network(Hu etal.,2023) and treat it as target.

My question is that since the GT is generated via another trimap-based matting method, meaning that's the highest quality you could get. Then, what's the purpose of your work ? If I want to get the high-quality matted mask, I can just use SAM with some refinement method to get high-quality instance mask, and then generated the T(x) using Line 236 and Line 241-244, then I can get better quality that your method.

In this way, why I need authors method ?

2. I don't see any results authors compared with (Hu etal.,2023) which is used to generate the GT for the training dataset. You can easily use your way to generate trimap and then run (Hu etal.,2023).

3. Performance Improvement Analysis section is the most important part in this paper especially to show the effectiveness of the proposed dataset. However, there is no summary of what is (a) (b) (c) )(d) means from the table. And there is almost no explanation on that part in the main paper.

4. Since the major novelty of this paper is the contributed dataset, Authors should do a comprehensive analysis on the diversity of dataset and comprehensive experiments why this dataset is helpful. Using your dataset and your new model to compare with a lot of other methods that trained on other dataset cannot tell what's the advantage of your dataset (basically your major novelty)

5. Authors should show results on green background to see how good your matting is. Showing mask only cannot tell much difference. Especially sharper mask or more matted hair doesn't mean your model is correct. This could only be seen from the green background composition.

i) Dataset

• Inaccurate annotations. The motivation to create the COCO-Matting is good, considering the natural and complex scenes, but I have two major concerns about the generation procedures of the annotations. First, I wonder whether the ‘Accessory Fusion’ step would still lead to cases where objects that do not correspond to the target instance are retained. For example, will the cup be retained if another human instance holds a cup that blocks part of the current instance's body? Also, in the lower left of Figure 2, if the baseball and the human instance are close enough, with a distance smaller than the dilation kernel, will the baseball be retained? Overall, I feel the fusion process is too heuristic and empirical, which may inevitably lead to uncertain annotation errors. I suggest the authors consider including manual post-screening to correct any potential errors. Another concern is in the ‘Mask-to-Matte’ step, the accuracy of the alpha mattes highly depends on the used trimap-based network, and I wonder whether it can meet the standard required for image matting annotation. As shown in Figure 1, the generated ‘GT’ alpha mattes do not seem to be good enough.
• Limitations of the natural scenarios and image resolution. The COCO-Matting mainly contains scenes with humans and does not adequately cover natural scenes relevant to matting, such as transparent or mesh materials, or any semantic category. It is OK to focus on human matting, but the dataset contribution seems overclaimed, so as the title of the paper. This work mainly deals with human-specific natural scenes but not all real scenarios in the wild. Another potential limitation is that matting generally requires high-resolution training data, but the COCO-Matting may not meet this prerequisite.

ii) Method

• The performance improvement comes at the cost of usability and flexibility. Compared with the MAM, Matting Anything, and SmartMat, the proposed method is specifically optimized for box prompts. It is not surprising to see the improvement under this specialized optimization. Despite the improvements in experiments, SEMat does not support other forms of prompts such as masks, points, and scribbles, reducing flexibility and usability in interactive image matting.
• Limited technical depth. The proposed enhancements of the SAM mainly involve off-the-shelf techniques such as LoRA and Trimap classification loss. These attempts are either generic or well-justified in the field. The fusion of the box prompt information at the input level, at the feature level, and at the output level is good, but not enough.

iii) Experiments

• Unfair comparison with SmartMat. Their backbones are different. How would the SEMat perform if replaced with DINOv2 as the backbone?
• The current experimental results may not support the SEMat as an effective, generic NATURAL image matting approach in the wild. The visualizations mainly focus on portraits and animals with similar hair/fur patterns. Aside from the cup shown in Figure 4, what is SEMat's performance on other natural images?
• It is not sufficiently convincing to compare the SEMat with the InstMatt under the current setting. The SEMat requires additional Bbox input from Grounding DINO for instance matting; some improvements may be attributed to the good localization performance. Also, conducting the ablation study on the HIM2k may not be a good choice. The ablation study should focus on justifying the matting ability of the model.
• Interactive matting with Bbox prompts has already been integrated into the latest version of Adobe Photoshop. Could it be compared with the proposed method?

iv) Minor issues

• There is a typo error in Line349, where ‘Specially, We…’ should be ‘Specially, we…’