Understanding Scale Shift in Domain Generalization for Crowd Localization

1. The paper frames the issue of scale shift as a new challenge within domain generalization for crowd localization, but this framing seems overstated. Scale shift, where different head sizes (scales) impact model performance across datasets, is not fundamentally new. Previous works have already explored the impact of scale variation on domain adaptation in crowd analysis (albeit under different terminologies), suggesting that this issue is more a subset of a well-studied generalization problem rather than a novel concept. Claiming it as the "first study" on "scale shift domain generalization" could be seen as an attempt to rebrand existing challenges without sufficient justification.

2. The proposed "Semantic Hook" technique to mitigate scale shift claims to enhance the association between semantic features and task predictions, but its practical effectiveness remains questionable. This method involves adding Gaussian noise to "hook" relevant features, yet the theoretical rationale behind this approach is underdeveloped. How "Semantic Hook" contributes to decoupling scale-related biases from semantic content is unclear. Additionally, the improvement in F1 scores presented in Table 2 is marginal, suggesting that the Semantic Hook might not be a robust solution.

3. While the paper provides a comparison of 20 domain generalization algorithms, there is little discussion about the practical differences in their robustness against scale shifts. The Semantic Hook’s performance is only marginally better than ERM, raising doubts about its practical value. Furthermore, the experiments rely heavily on F1 scores across ScaleBench domains but do not include additional evaluation metrics (e.g., precision, recall) that could provide a fuller picture of model performance under scale shift.

4. ScaleBench, with its scale-based domain partitions, may not accurately reflect real-world applications where scale distributions are more complex and continuous rather than discretely defined. The Leave-One-Out approach used for evaluation also artificially simplifies the generalization challenge. Real-world scenarios often involve more nuanced and diverse shifts between training and deployment environments, suggesting that the paper’s evaluation may lack external validity.

1- There is limited mention (with brief explanation for each) of papers explicitly in crowd counting/localization fields that tackle the issue of scale variance.

2- While the paper provides a comprehensive benchmark for scale-related domain generalization, it lacks coverage of crowd counting/localization methods specifically designed for domain generalization. How many of the methods in Table 2 discuss scale variance for crowd counting specifically?

3- The paper does not clarify what has been done to prevent overfitting, particularly given the possible complexity of the models relative to the training data provided.

4- Although Tables 6-18 and a brief discussion for each are included in the appendix, the paper lacks an in-depth analysis explaining why certain methods outperform others in specific cases. A discussion of these results would add valuable context to understand the strengths and limitations of each approach under different scale conditions. what could be the issue that each of these methods fail in generalizing to new domain?

The paper needs more detailed explanations regarding how Semantic Hook mitigates the scale shift in domain generalization. It also needs to clarify which variables or attributes are being generalized from the perturbation added during training. Additionally, the improvements from the proposed method on the ScaleBench benchmark are marginal compared to the baseline method. Furthermore, the mathematical formulations (Eq. 6) used for the theoretical analysis need to be corrected.

1. The introduction lacks highlighting core contributions and findings.
2. Although the authors indicate that the paper does not primarily focus on introducing a new method, the experiments in the main text feel somewhat limited.
3. The appendix contains several formatting issues, particularly with tables. Inconsistencies include varying font sizes, tables floating in the middle of pages, and some tables exceeding the page width. These layout problems affect readability and detract from the paper's presentation quality.

1. The rationality of ScaleBench is questionable. First, while the perspective effect often occurs in crowd localization, there exist images that are captured from different angles (e.g., top view). In such scenarios, image distribution regularization may fail to partition the images correctly. Second, in the real world, scale shift is often coupled with other factors, such as occlusion, weather, and appearance. For example, when the object suffers from significant appearance variations, the counting model may fail to localize objects even if training and testing data yield the same scale distribution. Third, dividing images into patches will inevitably result in incomplete objects, which could affect the localization results. Therefore, evaluations on ScaleBench may not rigorously reflect the influence of scale shift.
2. The proposed SemanticHook does not exhibit superiority over existing methods. As shown in Table 1, the simplest baseline ERM already achieves good results. The proposed method is not necessarily better than ERM.
3. Following the previous comment, the rationale of SemanticHook is not entirely convincing. Eq. 6 suggests that p(s, c, …) can lead to a spurious association between the output y and scale c. This term is a joint distribution of semantic s and scale c. However, the authors merely try to enhance the semantic association between semantic s and output y. Experimental results demonstrate that such a technique does not address scale shift effectively. Additionally, perturbing image is not a new idea, which is widely used in adversarial attack.
4. It appears that the influence of image interpolation is not rigorously quantified in Table 4. First, the implementation of Random Augmentation shall be modified according to different domains, i.e., the range of random scaling should be customized based on domain Tiny, Small, and Normal. Second, it is necessary to train the model using different source domains to identify the effect of image interpolation. The results on domain Big are insufficient to conclude that the benefits of image interpolation are modest.
5. Regarding training details. In practice, random scaling is commonly used to alleviate scale variations. As the authors use this technique to train the model, the reported results may not correctly reveal the effect of scale shift, because random scaling already simulates different scales.
6. The paper lacks evaluations on previous methods featuring multi-scale architecture, e.g., STEER. Evaluations on these methods are helpful in revealing whether previous methods can handle scale variations.