Towards Robustness of Person Search against Corruptions

Dear Reviewer ujfi,

We sincerely appreciate your insightful suggestion and are grateful for your positive remarks on the motivation and insights behind our paper.

The areas you mentioned are indeed crucial topics actively being researched in the fields. Given that collecting data in diverse and specific scenarios is costly and could raise ethical concerns, our benchmarks can serve as a scalable proxy for harsh environments across various situations. We agree that infrared and near-infrared scenarios represent practically important challenges in person retrieval applications, extending these corruption scenarios will provide significant value.

Recent studies have shown that well-designed multi-modal approaches show higher generalizability compared to single-modal methods [1,2,3], and exploring this topic in person retrieval research is a good extension option.

With the rising popularity of LLMs, obtaining rich textual annotations for images has become more feasible, computer vision fields are increasingly embracing multi-modal paradigms. Each modality can face different distinct corruption patterns, we believe robustness analysis in multi-modal scenarios provides valuable insights for this field as well. We are grateful for your insightful feedback, we will not stop here and are eager to explore these extensions.

References:

[1] CLIP: Learning Transferable Visual Models From Natural Language Supervision (ICML 21)

[2] BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (ICML 22)

[3] PLIP: Language-Image Pre-training for Person Representation Learning (NeurIPS 24)

Dear Reviewer ujfi,

In general, full-mark paper is likely to address a highly significant issue within a particular field, offering high-quality research and substantial contributions. This paper receives divergent reviews. Absolutely, other reviewers have provided constructive comments for the limitations and weakness of this paper. After carefully going through this paper, I believe the presentation and writing should be improved. Could you share your thoughts on why you believe this paper deserves a perfect score? Otherwise, it will receive an unethical flag for your review.

Thank you for your reminder. In fact, the score of 10 that I gave was based on two main reasons:

1. I had previously reviewed this paper for NeurIPS. In my last review, I raised several questions, and this time, upon receiving the paper again, I carefully examined the detailed content. I found that the core issues I raised were thoroughly addressed in the main paper, particularly in the final two pages, including the frequency analysis visualizations and the impact of foreground and background. As a result, I gave the paper a high score of 8 during the initial review.
2. Due to my limited reviewing experience, I was not entirely clear about the grading scale for a score of 10. Based on your feedback, I am willing to adjust the score to 8 as you suggested.

Dear Reviewer Yohg,

Thank you for constructive reviews and your time and effort in evaluating our work. Below, we address your concerns and questions.

About missing severity level description in the paper

We would like to refer to Lines 200-205 in Section 3.1 of the main paper, which provides an explanation of severity levels using rain corruption as a representative example.

Appendix C.3 contains comprehensive implementation details for all 18 corruption types, including control factors and their quantitative explanation of all parameter values constructing our framework.

From Appendix, we bring the description of how we define the severity using snow as an example. We adjust several attributes:

• Snow density: The heavier the snow, the more snowfall appears.

• Snow particle size: As the snow becomes heavier, the size of the snowflakes increases.

• Snow particle transparency: As the snowfall becomes heavier, there's typically more moisture in the air, leading to a higher probability of water droplets combining to form larger and more distinct snowflakes.

• Brightness of scene: Snow increases the brightness of darker areas in the image, simulating how snow reflects light.

• Slope of snow: This simulates how much the snow is tilted by the wind. At lower severity levels, the snowflakes are lighter in weight and thus more easily tilted by the wind, resulting in a steeper slope. At higher severity levels, the snow falls more vertically to the ground, as heavier snowflakes are less affected by wind.

We define the severity for person search task through data quality check, to ensure persons in the images are still detectable and re-identifiable by humans, even at the highest severity.

About adding more related works

Thank you for your suggestion.

We conduct the experiment of Tab 3, with three additional related works [9,10,11], while we were unable to include the other suggested model in our current rebuttal timeline due to code availability issue. We evaluate the model on the CUHK-SYSU and PRW, we validate our approach on this model as well.

We appreciate your feedback and will include the experiment results of [9,10,11] in our revision.

About the scalability of the method

As CUHK-SYSU and PRW are the two major datasets for person search, and the models we use are evaluated and validated on these two benchmarks, we use them to allow for comparison with existing evaluations on the same basis.

Our evaluation framework for corruption robustness is designed to be seamlessly adaptive to existing dataset.

To further validate our method's scalability, we conducted additional experiments on PoseTrack21[7], a large-scale dataset recently adopted in person search research [8]. The experimental results show that our method performs effectively on large-scale dataset as well.

About methodological innovation

Even though we leverage some existing techniques such as AugMix, we have designed our method tailored to the person search task, based on our experimental observations.

Our method, though relatively simple in its design, derives from our robustness analysis and observations of the person search framework, showing to be simple-yet-effective in its performance. To the best of our knowledge, we are the first to introduce selective augmentation and its corresponding regularization approach in the person search field.

Also, its simplicity enables its broad applicability across various person search models without incurring additional parameter or inference time costs.

With the advantage that our method maintains clean set performance within a ±2% range, it pushes the robustness performance of diverse state-of-the-art models to a large margin, where models themselves show a limited resilience to corruptions in Tab 1.

Nonetheless, we also agree that future technique-focused research, such as advanced architectures for robust person search, can also be conducted, which we believe is a promising future direction. We believe our comprehensive framework and analyses will serve as a valuable foundation for such future technical innovations in the field.

About the value of our benchmark beyond evaluation metrics

In the field of corruption robustness, the prevalent approach is to evaluate the model performance by adding noise to the image, which has contributed to establish the framework for robustness analysis across various computer vision fields (Yi et al. 2021; Chen et al. 2021; Kong et al. 2023), and it encourages continuous exploration in this direction [1,2 Lee et al. 2022].

Although our framework leverages some existing techniques, its design is specialized with task-specific insights for evaluating person search framework, beyond merely employing the existing approach.

• Our framework construction is guided by experimental observation. Based on the findings in Appendix A.1, we apply different corruptions to query and gallery images. As demonstrated in Appendix Table 6, applying identical corruptions to both query and gallery images could introduce bias by artificially increasing image similarity. This design also reflects the insight that query and gallery images may not always exhibit the same type or severity of corruption in real-world.

• We carefully select corruption types and calibrate their severity levels optimized for evaluating person search framework. In deciding types of corruption, we focus on real-world deployment scenarios while maintaining comprehensive coverage of corruption types. Given that noise-related corruptions in ImageNet-C have an overlapping score close to 1 [3], we replace shot-noise and impulse-noise with rain and dark, which better represent realistic deployment scenarios. We calibrate severity levels to be challenging while ensuring that persons in the images remain detectable and re-identifiable by humans even at the highest severity.

• Mitigating ethical concerns of new data collection: As the person search field demands careful ethical consideration due to its use of human data, our methodology provides an analysis framework containing diverse corruption scenarios without ethical concerns.

We would like to emphasize that even though we leverage the existing technique, our framework derive the first pioneering analysis on this topic, thus providing a comprehensive framework where the evaluations, analyses, and further methodological research can be encouraged.

We conduct experiments to further validate the value of our corruption benchmark. Instead of using clean data for training, we use our corruption benchmark for training and test the performance on 4 corruption scenarios -fog, dark, rain and blur- in BDD100K, a real-world corruption introduced in Tab 5. The results show training with our corruption benchmark helps improve the performance, exhibiting the value that our benchmark can bridge the distribution gap towards real world.

About the difference between augmentation and corruption

Data augmentation is a popular approach for enhancing model robustness and generalizability.

Data augmentation has been widely utilized in domain generalization[4,5], and it has been revealed by the survey that data augmentation is the most widely adopted[6], which lead us to pioneer the application of data augmentation for robustness in person search.

Distinct from general adoptions, leveraging our observations, we tailor the usage of the augmentation techniques to the person search framework by selectively applying them to the foreground regions.

Furthermore, our method shows the effectiveness not only on our corruptions, but also on the real corruption scenarios, as validated in Tab 5 of main paper.

About notations

We have fixed some typos in our submission. To clarify potential ambiguity in Eq 5, we have revised the notation $h$ to $h_c$.

We welcome the opportunity to provide further clarification if any points remain unclear.

[1] Robust Heterogeneous Federated Learning under Data Corruption (ICCV 23)

[2] Towards Better Robustness against Common Corruptions for Unsupervised Domain Adaptation (ICCV 23)

[3] Using the Overlapping Score to Improve Corruption Benchmarks (ICIP 21)

[4] Generalizing to Unseen Domains: A Survey on Domain Generalization (IJCAI 21)

[5] Domain Generalization: A Survey (TPAMI 23)

[6] A Survey on the Robustness of Computer Vision Models against Common Corruptions (Arxiv 23)

[7] PoseTrack21: A Dataset for Person Search, Multi-Object Tracking and Multi-Person Pose Tracking (CVPR 21)

[8] Divide and Conquer: Hybrid Pre-training for Person Search (AAAI 24)

[9] PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search (ACCV 22)

[10] Ground-to-Aerial Person Search: Benchmark Dataset and Approach (MM 23)

[11] SAT: Scale-Augmented Transformer for Person Search (WACV 23)

Dear reviewer Yohg,

As the reviewer-author discussion period is coming to a close, we kindly ask if there are any remaining concerns or points about our submission that we haven't sufficiently addressed. We're ready to provide additional clarifications or information if needed.

Once again, we appreciate your valuable efforts and feedback to strengthen our work.

Best regards,

Authors of Submission 7041

Thanks for your reply. What concerns me is that the "real-world" dataset in your work contains only 400 samples, which is far from sufficient to represent the full diversity of real-world scenarios. As a result, many potential corruptions that exist in practical applications are not covered in the benchmarks, raising questions about the generalizability of the proposed method. Apparantly, more real-world evaluation should be conducted. Also, the computational overhead of the method has not been discussed, which could pose significant challenges when deploying in real-time applications.

Dear Reviewer Yohg,

Thank you for your constructive feedback and for taking time to visit our rebuttal discussion.

About the computational overhead of our method when deployed in real-time applications

Our method does NOT introduce any computational cost in real-time applications, as both our foreground-aware augmentation and regularization are training techniques.

Without additional model parameters or real-time cost, our method can be seamlessly integrated into existing person search models, as shown in Tab 3 of the main paper.

About the scalability of validation result requiring more real-world evaluation

We understand your concern about the scalability of the result since the real domain dataset we construct are not extensively large.

Due to the requirement of obtaining multiple viewpoints of the same person, inherent ethical issues, and privacy concerns, extensive data collection in person search demands substantial costs with careful consideration, with these costs escalating further when extended to special domains such as corrupted environments.

In this context, for some challenging conditions of this field, small size datasets[12,13], which only include (150 / 600) and (420 / 600) number of images (#test split / #entire dataset), have contributed to advance the field.

Given these aforementioned challenges in this field, our approach of constructing corruption benchmarks can serve as an effective proxy for measuring robustness across diverse environments.

Since collecting new real-world data confronts rapidly escalating costs when attempting to cover comprehensive corruption scenarios, our framework offers the effective solution in this aspect, where the approach of establishing corruption environments within our framework has been extensively demonstrated across numerous computer vision fields [14,15,16,17,18,19,20].

Furthermore, our frequency sensitivity analysis in Figure 6 of the main paper demonstrates that applying our method to the baseline shows improving resilience to a wide range of frequency perturbations including low-, middle- and high-frequency. Yin et al. (2019) analyzed the common corruptions in the frequency domain, demonstrating that fog and contrast have relatively low-frequency components, noise-related corruptions have high-frequency components, and blur and pixelate have middle-frequency components. Given this comprehensive coverage of our method across the frequency spectrum, we believe our approach to be adaptable to unknown corruptions in other frequency ranges as well.

—

References:

[12] Cross-Domain Adversarial Feature Learning for Sketch Re-identification (MM 18)

[13] Partial Person Re-identification (ICCV 15)

[14] When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks (CVPR 21)

[15] Robustness Disparities in Face Detection (NeurIPS 22)

[16] Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving (CVPR 23)

[17] RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions (NeurIPS 23)

[18] Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models (NeurIPS 23)

[19] Robo3D: Towards Robust and Reliable 3D Perception against Corruptions (ICCV 23)

[20] Benchmarking the Robustness of Cross-view Geo-localization Models (ECCV 24)

About design principles of our benchmark and method (Answer for ‘The strength of this paper lies in …’)

About the benchmark:

In the field of corruption robustness, the prevalent approach is to evaluate the model performance by adding noise to the image, which has contributed to establish the framework for robustness analysis across various computer vision fields (Yi et al., 2021; Chen et al., 2021; Kong et al., 2023), and it encourages continuous exploration in this direction [1,2, Lee et al., 2022].

Although our framework leverages some existing techniques, its design is specialized with task-specific insights for evaluating person search framework, beyond merely employing the existing approach.

• Our benchmark construction is guided by experimental observation. Based on the findings in Appendix A.1, we apply different corruptions to query and gallery images. As demonstrated in Appendix Table 6, applying identical corruptions to both query and gallery images could introduce bias by artificially increasing image similarity. This design choice also reflects the insight that query and gallery images may not always exhibit the same type or severity level of corruption in real-world scenarios.

• We carefully select corruption types and calibrate their severity levels optimized for evaluating person search framework. In deciding types of corruption, we focus on real-world deployment scenarios while maintaining comprehensive coverage of corruption types. Given that noise-related corruptions in ImageNet-C have an overlapping score close to 1 [3], we replace shot-noise and impulse-noise with rain and dark corruptions, which better represent realistic deployment scenarios. We calibrate severity levels to be challenging while ensuring that persons in the images remain detectable and re-identifiable by humans even at the highest severity. In Appendix C.3, we provide detailed documentation of all control factors and their corresponding parameters used to define severity levels for all 18 corruptions.

• Mitigating ethical concerns of new data collection in person search, our benchmark serves as a proxy for diverse scenarios . As the person search field demands careful ethical consideration due to its use of human data, our methodology provides an analysis framework containing diverse corruption scenarios without ethical concerns of new data collection.

We would like to emphasize that even though we leverage the existing technique, our benchmarks derive the first pioneering analysis on this topic, thus providing a comprehensive framework where the evaluations, analyses, and further methodological research can be encouraged.

About the proposed solution:

Our proposed solution, though relatively simple in its design, derives from our robustness analysis and observations of the person search framework, showing to be simple-yet-effective in its performance. Also, its simplicity enables its broad applicability across various person search models without incurring additional parameter or inference time costs. With the advantage that our method maintains clean set performance within a ±2% range, it pushes the robustness performance of diverse state-of-the-art models to a large margin.

—

References:

[1] Robust Heterogeneous Federated Learning under Data Corruption (ICCV 23)

[2] Towards Better Robustness against Common Corruptions for Unsupervised Domain Adaptation (ICCV 23)

[3] Using the Overlapping Score to Improve Corruption Benchmarks (ICIP 21)

Dear Reviewer d5hd,

Thank you for your thorough review and constructive feedback. We appreciate your time and effort reviewing our work. Below, we address your concerns.

About real-world relevance of our corruptions

About data augmentation(corruption) to simulate real-world disturbances, during the rebuttal period, we conduct additional experiments to further validate the effectiveness of our corruption benchmark. Instead of using clean data for the model training, we use our corruption benchmark for model training, and evaluate the performance on the four corruption scenarios -fog, dark, rain and blur- in BDD100K, a real-world corruption dataset as introduced in Tab 5 of the main paper. The experimental results show that training with our corruption benchmark helps improve the performance, verifying that our benchmark can serve as an effective proxy for the real world by bridging the distribution gap.

cf. About data augmentations used in our method, our method shows performance improvement on real-world corruption scenarios, as validated in Table 5 of the main paper.

About foreground-aware augmentation

During the rebuttal period, we provide a deeper exploration of our foreground-aware augmentation with the qualitative analysis and Grad-CAM visualization, as shown in the Figure 8b of Appendix B.5.

We compare the content in bounding box regions after augmentation applied based on two regional criteria: entire image (1st column) and foreground region (2nd column).

Naive adoption of augmentation by applying it to the entire image (1st column) could lead to unreliable bounding boxes, as observed in the 1st column where bounding boxes miss some discriminative parts of a person.

In contrast, with foreground-aware strategy (2nd column), most discriminative parts of the person remain within the bounding box while augmentation is successfully applied. Preserving discriminative information within the bounding box is crucial, as it serves as the criterion for selecting region proposals used in training person search models. Accordingly, the results of foreground-aware strategy in 5th column show that the model better captures the person's discriminative information (4th/5th column: without/with applying foreground-aware strategy).

Data augmentation has proven to be an effective approach in various computer vision fields,

But the adoption of data augmentation in person search remains relatively unexplored, as the naïve use could lead to the problem as shown in the 1st column of Figure 8b (Appendix) or Figure 5 (Main paper), such as severe semantic perturbation and unreliable bounding boxes, which lead us to pioneer the application of data augmentation for robustness in person search.

Although this technique might seem a bit incremental, we pioneer the usage of data augmentation by tailoring it to the person search task, based on our experimental observations.

To the best of our knowledge, we are the first to introduce the selective augmentation strategy in the person search field.

This technique, though relatively simple in its design, by leveraging our observations, shows simple yet effective performance. It has the advantage of broad applicability for various person search models without introducing additional parameters or inference time cost.

Question about our experimental observations

Thank you for your insightful question.

We had measured both detection and recognition performance, as illustrated in Figure 4a of the main paper. As shown in the figure, the recognition performance experiences more significant degradation compared to detection performance.

Furthermore, we would like to refer to Appendix A.3, which provides a clear answer to your question.

We simultaneously conduct two experiments that led to our observations: Individual evaluation of detection and reID stage, and evaluation where corruption is applied to either the background or the foreground, which can be categorized into four testing scenarios:

• fg-corrupted & Representation against Corruption
• fg-corrupted & Detection against Corruption
• bg-corrupted & Representation against Corruption
• bg-corrupted & Detection against Corruption

The results show that this experiment also exhibits consistent results with our observations, leading to the conclusion that the representation stage is particularly vulnerable when damage is applied to the foreground as well.

Dear Reviewer d5hd,

As the reviewer-author discussion period is coming to a close, we kindly ask if there are any remaining concerns or points about our submission that we haven't sufficiently addressed. We're ready to provide additional clarifications or information if needed.

Once again, we appreciate your valuable efforts and feedback to strengthen our work.

Best regards,

Authors of submission 7041

I have reviewed the comments from the other reviewers, and overall, the proposed new task in this paper is of practical significance. However, there are still several issues that need to be addressed, which is why I have decided to lower my score:

1. Limited Contribution of the Dataset: The proposed dataset is simply an augmented version of an existing dataset and does not accurately simulate real-world scenarios. In fact, the test dataset should be sourced from real-world environments in order to effectively validate the actual performance of the proposed method.

2. Limited Innovation of the Method: Both I and reviewers Yohg and tUry pointed out in our initial evaluations that the innovation of the proposed method is relatively weak. Furthermore, since the dataset does not simulate real-world conditions, the performance validation of the proposed method is insufficient.

3. Inconsistent Reviewer Evaluations: Reviewer ujfi gave the paper a perfect score and claimed there are no obvious weaknesses, which clearly contradicts the opinions of the other reviewers. For example, reviewer Yohg pointed out that the manuscript contains numerous typographical errors, which are basic issues.

Overall, while I believe the proposed new task has practical significance, the authors' efforts have not sufficiently supported this task, and as a result, I am inclined to lower my score. The authors should further improve the construction of the dataset and the design of the method.

About our benchmarks not sourced from actual real environment (Answer for 1)

We understand your concern that we should source the test dataset from the real world to support the proposed new task. Indeed, real-world data collection has its own advantages.

In the current landscape, when we want to validate if our person search algorithm is robust in the real world, we need to pay high costs in order to collect the test dataset, while confronting unique challenges inherent to the data person search requires: privacy issues. This challenge becomes more complex when attempting to cover various real-world scenarios. Furthermore, gathering such diverse real-world scenarios from various locations and data collection environments becomes infeasible.

In this context, our framework provides a cost-effective, diverse covering, scalable solution, which is free from privacy issues.

Our approach of establishing corruption environments by curating the existing data has been extensively demonstrated across numerous computer vision fields [4,5,6,7,8,9,10].

Moving beyond simply augmenting, our proposed benchmarks are specially tailored with task-specific insights for evaluating person search framework, as outlined in the above comment.

As shown in the experiment of the above comment, our benchmark can narrow the distribution gap towards the real world, validating it to serve as an effective proxy for the real world.

• Diverse scenario coverage: Due to the high costs of real data collection, comprehensive scenario coverage remains limited in this field. To our best knowledge, no existing real dataset encompasses as many as 18 different scenarios. In contrast, our framework provides 18 arrays of corruption scenarios, firstly enabling robustness measurement of person search models across diverse corruption environments, encouraging robustness evaluations, analyses, and further methodological improvements.
• Scalable: When collecting the corruption scenarios through real world data collection, each collection becomes dependent on the specific situation of the location where it was collected. However, our framework is scalable and can be combined with other existing datasets collected in various ways from different situations, as verified in the experiment with larger scale PoseTrack21[11] dataset, conducted by the request of Reviewer Yohg.
• Avoiding privacy issues: As the person search field demands careful ethical consideration due to its use of human data, our methodology provides an analysis framework containing diverse corruption scenarios without introducing privacy concerns of new data collection. It implies that introducing our benchmark constructing approach to the person search field is especially valuable.

Although our approach may not perfectly mirror real-world environments, we believe our approach brings a meaningful research value by providing cost-effective, comprehensive, scalable, ethics-issue-free solutions to the field. As our work is the beginning of the branch of corruption robustness in person search, we believe our approach will serve as good preliminary work for future research, including future real-world data collection endeavors.

References:

[4] When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks (CVPR 21)

[5] Robustness Disparities in Face Detection (NeurIPS 22)

[6] Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving (CVPR 23)

[7] RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions (NeurIPS 23)

[8] Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models (NeurIPS 23)

[9] Robo3D: Towards Robust and Reliable 3D Perception against Corruptions (ICCV 23)

[10] Benchmarking the Robustness of Cross-view Geo-localization Models (ECCV 24)

[11] PoseTrack21: A Dataset for Person Search, Multi-Object Tracking and Multi-Person Pose Tracking (CVPR 21)

Dear Reviewer d5hd,

We appreciate your constructive feedback and for taking time to visit our rebuttal discussion.

Multifaceted reviewer guidance has largely strengthen our work (Answer for 3)

We have addressed the typographical ambiguities and incorporated these changes in the revised paper. We remain open to enhancing notations and clarifications if necessary, and would appreciate any future suggestions.

On the other hand, we are pleased that our presentation has facilitated easy understanding for readers, as recognized by positive comments from other reviewers such as "solid work" (tUry) and "easy to understand" (tUry, Yohg, ujfi).

Thanks to all reviewers for taking the time to improve our work, our work has become even stronger,

• by providing robustness analyses of additional six recently proposed models (Reviewer FKyP),
• by verifying the effectiveness of proposed method on three recent models (Reviewer Yohg),
• by validating the scalability of our framework (Reviewer Yohg),
• by validating the effectiveness of the proposed corruptions (Reviewer FKyP),
• by verifying the validity of our observations with the experiments in Tab 7 and with pretraining (Reviewers d5hd, tUry),
• by further exploring our method's intuition through CAM analysis (Reviewer d5hd).

Aligning with Reviewer ujfi's insight, we believe that our pioneering endeavor and framework, grounded in our motivations and analyses acknowledged by Reviewer FKyP, will serve as a starting point and stimulate future explorations in corruption robustness, including real world data collection and advanced methodology.

About the value of our method (Answer for 2)

Before starting, we would like to notify that the development of our robustness solution was made possible thanks to the foundations for robustness evaluation and analyses our benchmarks provide.

We understand your concern that its design of our method might appear relatively simple.

Though our solution might look simple, it was derived from our observations of the person search framework, leading to show a simple yet effective performance.

Thanks to its simplicity, it can be seamlessly applied to person search models without worrying about additional parameters or real-time costs. Consequently, our method can significantly strengthen the robustness of various existing well-designed person search models, where the models themselves show a limited resilience to corruptions in Table 1 of the main paper.

Even though we leverage some existing techniques such as AugMix, we have designed our method tailored to the person search task, based on our experimental observations. As a result, our method can strengthen the robustness while maintaining clean set performance within ±2% range.

We provide evidence in various ways validating the generalizability of our method.

• We validate the generalizability of our method across four real world corruption scenarios, as shown in Table 6 of the main paper.
• Our frequency sensitivity analysis in Figure 6 of the main paper demonstrates that applying our method to the baseline shows improving resilience to a wide range of frequency perturbations including low-, middle- and high-frequency. Yin et al. (2019) analyzed the common corruptions in the frequency domain, demonstrating that fog and contrast have relatively low-frequency components, noise-related corruptions have high-frequency components, and blur and pixelate have middle-frequency components. Given this comprehensive coverage of our method across the frequency spectrum, we believe our approach to be adaptable to unknown corruptions in other frequency ranges as well.
• Extensive experiments with five different person search models validate the effectiveness of our proposed method.

Nonetheless, we also agree that future technique-focused research, such as advanced network architectures for robust person search, can also be conducted, which we believe is a promising future direction. We believe our comprehensive framework, benchmarks, and analyses will serve as a valuable foundation for such future technical innovations in the field.

Dear Reviewer tUry,

Thank you for your insightful comments and positive remarks on our paper's structure and contributions. We appreciate your feedback and the requested interesting experiments. We have elaborated on the points or questions raised in your review below.

Question about code, model, and dataset availability

Yes! Like mentioned in the paper, we will release our complete framework including code, model, and dataset.

Question about additional experiments

We appreciate your insight to suggest the interesting experiment, which also interests us.

We conduct the experiment with pretrained methods mentioned above.

As you anticipated, in most cases, pretrained initializations with good person reID representations contribute to measurable improvements, which also aligns with our findings. Thanks to your suggested experiment, it confirms the potential of enhanced representation ability from pretrained knowledge to improve robustness, a more salient approach to enhance the representation utilizing external knowledge (e.g. [1]) can be valuable to explore than current naïve utilization (model initialization).

In our experiment, we utilize pretrained ResNet50 models from LUP, LUP-NL, PLIP, CION, while we were unable to include UPReID and ISR in our current rebuttal timeline due to code availability issues or missing ResNet weights. Like our paper's experimental setup, we use SeqNet as the baseline model and conduct experiments on CUHK-SYSU and PRW datasets.

We will revise our paper to reflect additional experiments and discussions.

Although the entire work is very solid, it relatively lacks …

Thank you for your careful review and insightful feedback on our work.

We would like to emphasize that one of our main contributions is the first pioneering analysis on this topic, and providing a comprehensive framework where the evaluations, analyses, and further methodological research can be encouraged.

Our method, though relatively simple in its design, derives from our robustness analysis and observations of the person search framework, showing to be simple-yet-effective in its performance. Also, its simplicity enables its broad applicability across various person search models, as shown in Table 3 (main paper). With the advantage that our method maintains clean set performance within a ±2% range, it pushes the robustness performance of diverse state-of-the-art models to a large margin, where the models themselves show a limited resilience to corruptions in Table 1 (main paper).

Even though we leverage some existing techniques such as AugMix, we have designed our method tailored to the person search task, based on our experimental observations. To the best of our knowledge, we are the first to introduce selective augmentation and its corresponding regularization approach in the person search field.

Nonetheless, we also agree that future technique-focused research, such as advanced network architectures for robust person search, can also be conducted, which we believe is a promising direction for future work. We believe our comprehensive framework, benchmarks, and analyses will serve as a valuable foundation for such future technical innovations in the field.

References:

[1] Making person search enjoy the merits of person re-identification (PR 2022)

Dear Reviewer tUry,

Thank you for taking the time to consider our responses and for raising the score. We are pleased that our responses have answered your questions.

Your interesting experimental suggestion has enabled us to improve the paper and verify the findings of our work. We will include the additional experimental results and discussions in our revision.

If you have any remaining questions or feedback, we would be glad to provide additional information or results. We appreciate your insightful review and your time in improving our work.

Best regards,

Authors of submission 7041

About the value of our benchmark beyond dataset processing

In the field of corruption robustness, the prevalent approach is to evaluate the model performance by adding noise to the image, which has contributed to establish the framework for robustness analysis across various computer vision fields (Yi et al., 2021; Chen et al., 2021; Kong et al., 2023), and it encourages continuous exploration in this direction [6,7, Lee et al., 2022].

Although our framework leverages some existing techniques, its design is specialized with task-specific insights for evaluating person search framework, beyond merely employing the existing approach.

• Our benchmark construction is guided by experimental observation. Based on the findings in Appendix A.1, we apply different corruptions to query and gallery images. As demonstrated in Appendix Table 6, applying identical corruptions to both query and gallery images could introduce bias by artificially increasing image similarity. This design choice also reflects the insight that query and gallery images may not always exhibit the same type or severity level of corruption in real-world scenarios.

• We carefully select corruption types and calibrate their severity levels optimized for evaluating person search framework. In deciding types of corruption, we focus on real-world deployment scenarios while maintaining comprehensive coverage of corruption types. Given that noise-related corruptions in ImageNet-C have an overlapping score close to 1 [8], we replace shot-noise and impulse-noise with rain and dark corruptions, which better represent realistic deployment scenarios. We calibrate severity levels to be challenging while ensuring that persons in the images remain detectable and re-identifiable by humans even at the highest severity.

• While avoiding ethical concerns in person search, our benchmark serves as a proxy for diverse scenarios . As the person search field demands careful ethical consideration due to its use of human data, our methodology provides an analysis framework containing diverse corruption scenarios without ethical concerns of new data collection.

We would like to emphasize that even though we leverage the existing technique, our benchmarks derive the first pioneering analysis on this topic, thus providing a comprehensive framework where the evaluations, analyses, and further methodological research can be encouraged.

During the rebuttal period, we conduct additional experiments to further validate the effectiveness of our corruptions. We conduct the experiments on four corruption scenarios: dark, rain, blur, and fog. We first train our model on each proposed corruption corresponding to four scenarios, subsequently, we evaluate the model on BDD100K, a real-world corruption dataset as referred to in Tab 5 of the main paper.

The experimental results show that our proposed corruptions can bridge the distribution gap with real corruption scenarios, serving as an effective proxy for the real world.

References

[1] Efficient Person Search: An Anchor-Free Approach (IJCV 23)

[2] Ground-to-Aerial Person Search: Benchmark Dataset and Approach (MM 23)

[3] PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search (ACCV 22)

[4] SAT: Scale-Augmented Transformer for Person Search (WACV 23)

[5] Anchor-Free Person Search (CVPR 21)

[6] Robust Heterogeneous Federated Learning under Data Corruption (ICCV 23)

[7] Towards Better Robustness against Common Corruptions for Unsupervised Domain Adaptation (ICCV 23)

[8] Using the Overlapping Score to Improve Corruption Benchmarks (ICIP 21)

Dear Reviewer FKyP,

Thank you for your insightful reviews and for taking the time to evaluate our work. Below, we address your concerns.

About extended evaluation with recently proposed methods

During the rebuttal period, we evaluate the corruption robustness of six recently proposed models, hoping that our work will contribute to the development of this field.

We will include the additional experimental results in our revision.

About foreground-aware augmentation

Thank you for your careful review and insightful feedback on our work.

Although this might seem as common sense, we are the first to leverage this insight as methodological application in the person search field, to our best knowledge.

The adoption of data augmentation in person search remains relatively unexplored, as the naïve use can lead to the problem as shown in Fig 4 main paper, such as severe semantic perturbation and unreliable bounding boxes, which lead us to pioneer the application of data augmentation for robustness in person search.

This technique, though relatively simple in its design, is derived from our robustness analysis and observations of the person search framework, which makes it simple yet effective.

Also, its simplicity enables its broad applicability across various person search models without incurring additional parameter or inference time costs.

Thanks for the detailed response! My previous concerns are well addressed and I would like to raise my rating to accept.