Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning
摘要
评审与讨论
This paper presents a two-stage method to address the adverse weather problem in crowd counting. In the first stage, a multi-queue MoCo is established to capture crowd features in different weather conditions in an unsupervised manner. In the second stage, a refining module is placed before the density decoder to transform crowds in adverse weather conditions into normal conditions, resulting in improved counting performance in adverse weather conditions.
优点
- An unsupervised way is designed based on MoCo to extract weather condition prosperity in crowd image.
- A refiner is learned to transform images in abnormal weather into normal conditions for better prediction.
- The NWPU-Weather dataset is extracted from the NWPU-Crowd dataset for this new task.
缺点
- No visualization is provided for the unsupervised results. Whether the model works as expected is unclear.
- How the multi-queue MoCo is trained is a little unclear. see questions.
问题
- Could the authors provide some images belonging to different queues in the multi-queue MoCo?
- What is the strategy for assigning Q to the corresponding sub-queue? How is the relationship between the current Q and different sub-queues modeled?
- What is the difference between Eq. (2) and the vanilla MoCo? How does the inclusion of different sub-queues affect this loss function?
- can it be ensured that different sub-queues contain crowds from different weather conditions? Since this is an unsupervised approach, it can be assumed that images with similar properties could be assigned to the same category. The question is whether the properties learned from the multi-queue MoCo represent weather information.
- The SRR stage requires a label for the normal condition. How is this label defined based on the sub-queues?
- The GCC[1] dataset contains weather information (7 weather types) for each image. Perhaps the authors could analyze their unsupervised WRL on this dataset. Specifically, they can directly apply their method to this dataset and compare the unsupervised results with the provided ground truth to demonstrate whether the model performs as expected.
This paper proposes a contrastive learning-based method to deal with class imbalance for different weathers in crowd counting. A synthetic dataset with different weather augmentation is proposed based on. NWPU-Crowd. The unsupervised contrastive learning with multi-queue MoCo is proposed to learn better features for imbalanced classes with adverse weather. Then, a supervised contrastive learning is proposed to refine the learned feature.
优点
The proposed method achieves good improvement based on adverse weather.
缺点
- Section 3.2 presents weather-aware representation learning. However, it is unclear why contrastive learning can be used to learn weather-aware representation learning.
- The proposed method is not compared with other class imbalance strategies which is not convincing.
- The presentation is unclear. For example, in the caption in Figure 2, WRL is an unsupervised contrastive learning method while in Section 2, a supervised loss is used during training, which is confusing.
问题
- The method is a generic method to deal with data imbalance. Is it effective for other applications with data imbalance issues?
- Is multi-queue MoCo part of WRL or SRR? From Figure 2, multi-queue is part of SRR, but as described in the main text, it’s part of WRL.
- How is the generalization ability of the model trained on synthetic data? If the model trained on synthetic data can't be generalized to the real dataset, the contribution of the synthetic dataset is limited.
This paper focuses on crowd counting under extreme and adverse weather conditions. Specifically, the authors propose a two-stage framework, where the first stage introduces a multi-queue MoCo contrastive learning strategy to tackle the problem of weather class imbalance, and the second stage adopts the supervised contrastive loss to guide the refinement process. Extensive experiments demonstrate the effectiveness of the proposed method.
优点
This paper is well written and the motivation is clear. The proposed method can effectively improve the robustness of the adverse weather conditions.
缺点
I am not sure of the importance of this task. Since this topic is too small, it might not be interesting for the researcher in computer vision, even in crowd counting. Considering the high bar of ICLR, I do not think this task can match the ICLR.
What about simply using the existing image restoration methods to make pre-processing? I think the authors should have a discussion.
The experiments are not convincing. First, as shown in Tab.2, compared with previous methods, the performance of adverse samples is improved by sacrificing the performance of a portion of normal samples. Second, the authors should report the average performance of the whole dataset, not only normal and adverse. Third, compared with previous SOTA, the proposed method does not have significant performance gain.
What about the cross-domain performance? e.g., training on JHU-Crowd and testing on NWPU-Crowd weather.
问题
see weakness
The paper was reviewed by 3 experts. The major concerns were the narrow scope of the problem, unconvincing experiments, and lack of simple baselines (pre-processing with image restoration), unclear motivation/justification of the proposed method. There was no response submitted. The AC agrees with the comments, and thus recommends reject.
为何不给更高分
The concerns are valid, and there was no author response.
为何不给更低分
n/a
Reject