We sincerely thank Reviewer FsmM for the positive and constructive review. We are grateful for your recognition of our sufficient experiments, proposed benchmark, and the presentation. Below, we respond to your thoughtful questions and suggestions in detail.

---

Q1: "Please discuss whether there is a conflict between learning the complementary information of modalities for fusion queries and fully mining the effective information for single modality queries, and if there is, how to conduct the trade off?"

A: Thank you for your insightful comments.

In fact, there is a potential conflict between them: On one hand, to achieve complementarity in multimodal fusion (e.g., semantic-visual complementarity between texts and sketches), the model needs to learn cross-modal aligned representations, which may lead to the "dilution" of modality-specific information (e.g., unique color textures in paintings or night features in infrared images). On the other hand, over-emphasizing effective information mining for single modalities may widen feature space differences across modalities, hindering alignment during fusion and weakening complementarity.

When designing our method, we took this conflict into account and achieved a trade-off through the below strategies:

• Multi-modal Tokenizing Assembler. To encode data from different modalities into a shared embedding space, we designed a straightforward multi-modal tokenizing assembler. It consists of five lightweight sub-tokenizers, which are used to independently tokenize RGB, IR, CP, SK, and text data respectively. This operation can, to a certain extent, preserve the differences and uniqueness of the specific information of each modality.

• Multi-Expert Router. We utilized a shared encoder to extract modality-shared features. Meanwhile, to extract modality-specific features, we designed this multi-expert routing mechanism. This mechanism allocates a respective LoRA-based expert to each modality, promoting the mining of independent and effective information for each modality and ensuring the discriminability of features from different modalities.

• Feature Mixture. To achieve the fusion and interaction of complementary information between different modalities, we utilized this efficient feature mixture to fuse features from different modal combinations. The input of this mixture is the features of each modality with independent information, and the output is the multimodal fused feature.

In our training: for single-modal retrieval, we directly use independent features of each modality for alignment; for multimodal combined retrieval, we use fused features after mixture. This approach balances learning multimodal complementary information and mining effective single-modal information to some extent, as verified by ablation experiments in Table 2.

Additionally, we propose that future research in this area could focus on a "query type-aware mechanism". This would enable the model to dynamically adjust feature extraction focus based on whether the input is a single-modal or multimodal query, balancing the two more precisely. We will pursue this direction in subsequent work.

We will follow your suggestions and add this part of the discussion to the supplement in the future version.

---

Q2: "Please report the the space complexity, and the time cost of training stage and inference stage in different baselines."

A: Thank you for your suggestion. We report the space complexity (parameters), training time cost, and single inference forward time of each baseline in the table below.

The listed parameters include all module parameters of each baseline model during training, some of which may be discarded during inference. Training time is based on a unified setting of 60 epochs.

For inference time, to ensure efficient and fair comparison, a single forward inference is defined as: the model sequentially infers one sample of RGB, infrared, painted, sketch, and text, then fuses the features of the latter four into a multimodal feature (consistent with the quad-search process in the benchmark). This process was repeated 1000 times, with the average taken as the result.

As shown in the table, ReID5o not only maintains significantly leading retrieval performance but also has advantages in terms of parameter count, training time, and inference speed.

Following your suggestions, we will include comparisons of results in this regard in the supplement of future versions.

---

Q3: "When applying this foundation model on ReID tasks, would more parameters achieve better performance?"

A: When increasing the parameters of our model (especially the pre-trained multi-modal encoder within it), it can indeed achieve better performance on the ReID task. To verify this, we replaced the original CLIP-B/16 encoder in ReID5o with CLIP-L/14, conducted training under the same settings, and tested its performance on ORBench. The results are shown in the following table.

Considering that the ReID task is essentially a fine-grained representation learning task, we believe that the performance improvement brought by the increase in the number of parameters is mainly attributed to the stronger and more robust representation ability of larger-scale pre-trained multimodal encoders. This phenomenon has also been demonstrated in many other papers.

---

Q4: "The authors removed unclear samples from constructed dataset, but didn't literally or empirically analysis the robustness of the proposed model against possible noise in the real world. The possible approaches of enhancing the robustness of the model to real-world noise should be discussed in detail in A.7."

A: Thank you for your insightful comments.

First, please allow us to elaborate on two reasons why we removed unclear samples from the existing datasets:

• Clear, high-quality RGB samples aid in annotating accurate, feature-rich data across color paintings, sketches, and text. Since we rely on original RGB images to draw and describe individuals, unclear ones hinder the creation of high-quality, appearance-consistent data—making it hard to fully and accurately capture people's appearance features.

• Many prior studies indicate that high-quality data enhances representation learning, and filtering unclear samples directly improves data quality. For instance, unlike CLIP, which trains on noisy samples, BLIP uses a data filtering mechanism to remove poorly matched image-text pairs—achieving better performance with fewer training samples.

We acknowledge that the aforementioned sample removal measures will make our high-quality benchmark somewhat fail to reflect real-world noises, which is discussed in our A.7 Limitation.

However, our cross-dataset generalization experiments in Table 7 validate the robustness of our proposed method, as most datasets here contain obvious noise (e.g., CUHK-PEDES has many rough, identity-mismatched text descriptions). The results show that training on our ORBench yields better generalization than other datasets, with our ReID5o method also exhibiting significant superiority. This indicates that ORBench and ReID5o complement each other to enhance robustness against real-world noise.

Naturally, improving robustness against real-world noise remains an urgent research focus. We believe the following directions are worthy of research:

• Physics-inspired noise modeling and synthesis. The generation of real-world noises often follows physical laws, but existing data augmentation methods may deviate from the real distribution. We can construct noise models that conform to physical laws to better generate realistic noisy data. For example, by simulating the scattering effect of rain and snow on RGB images, we can enhance the model's robustness to weather-related noises.

• Dynamic noise-aware multimodal fusion. In real-world multimodal person ReID applications, different modal data are affected by noise to varying degrees. How to dynamically allocate fusion weights based on the uncertainty of modal noise to obtain robust multimodal representations is also a research direction worth exploring.

• Progressive denoising curriculum learning. By imitating the human learning process, we can first let the model learn basic representations from relatively clean data, and then gradually introduce noisier data. Alternatively, we can design some mechanisms to first recover a clean representation and then perform matching based on it, forcing the model to learn invariant representations in the presence of noise.

Following your suggestion, we will discuss the robustness to real-world noise detailedly in Section A.7 in our revised version.

---

Q5: "The computing cost is relatively large, which may limit its faster and wider application."

A: Thank you for your concern regarding the computing cost. We have taken it into account during the model design process, utilizing a lightweight multimodal tokenizing assembler, multi-expert router, feature mixture, and a shared encoder. For the results and discussions on this part, please refer to the Q2.

---

Last but not least, we would like to sincerely thank Reviewer FsmM again for the valuable time and constructive feedback provided during this review.

We sincerely thank Reviewer akx3 for the positive and encouraging review. We are grateful for your recognition of our ORBench dataset, ReID5o method, extensive experimental results, and the quality of writing. Below, we respond to your thoughtful questions and suggestions in detail.

---

Q1: "The RGB and infrared modality data in this paper are filtered from existing datasets. If the authors could independently collect new data for these modalities, it might have greater community influence."

A: The RGB and infrared data of our ORBench are manually filtered from the existing datasets. Then, through significant efforts, we have additionally expanded three modalities: color painting, sketch, and text.

Since collecting a large amount of real-world video data containing both RGB and infrared information of the same person takes a long time and involves some privacy issues, we have decided to use the existing datasets LLCM and SYSU-MM01 for these two modalities, without affecting the core purpose of the research.

We have obtained permission for the use and expansion of these two datasets through written communication with their creators, which complies with relevant regulations and requirements.

Of course, as you mentioned, collecting new RGB and infrared data ourselves will indeed increase the diversity of data in the field. We will further improve our work in this regard in future research.

---

Q2: "To ensure data quality, the data construction process in this paper appears to require substantial manual involvement, which somewhat limits the scalability of the dataset."

A: Considering that the development of existing technologies, especially those for fine-grained image caption generation and artistic style painting targeting persons, is still immature, we have indeed invested significant manual involvement. This is to maximize the quality and diversity of the dataset, minimize noise contamination within it, and ensure that data across various modalities can accurately depict a person from multiple aspects. For further details on the efforts we have made in constructing the dataset, please refer to our response to the second comment from Reviewer h3Ed.

Certainly, as you have mentioned, we also believe that large-scale expansion of the dataset is equally important. It will enable models to learn more robust features and facilitate practical application deployment. We will strive to achieve this in our future research endeavors.

---

Q3: "The painting styles in ORBench seem somewhat monolithic. Enhancing the diversity of painting styles might better improve generalization ability in real-world scenarios."

A: Certainly, using more different AI painting tools would enhance the diversity of painting styles, which, to a certain extent, would be more conducive to generalization in real-world scenarios. However, most AI art painting tools tend to produce hallucinations and perform poorly when generating paintings based on low-resolution person RGB images. After multiple investigations, we had to select the only AI tool that meets the requirements, namely Doubao Art Painting.

To increase the diversity of paintings, we have made the following efforts: On the one hand, during the painting process, we manually required and supervised the model to paint the same person from three perspectives, thereby introducing perspective diversity; on the other hand, we generated multiple samples for the same person, with 18 color paintings and 18 sketches per person. Due to the inherent randomness in AI generation, this also brings about a certain degree of sample diversity.

Our aforementioned efforts have increased the diversity of the dataset to a certain extent, and the experiments in Table 7 have also demonstrated that our ORBench has better generalization ability compared to other datasets. We will attempt to use more AI painting tools in the future to enhance the diversity of painting styles in the dataset.

---

Q4: "In the process of constructing the ORBench dataset, it seems that you filtered out a large number of low-quality RGB images from existing datasets. What was your motivation for doing so? Because low-quality photos often appear in real-world ReID applications."

A: The question you raised has been addressed in our response to Reviewer FsmM's fourth question. We have included it below for your convenience.

First, please allow us to elaborate on two reasons why we removed unclear samples from the existing datasets:

• Clear, high-quality RGB samples aid in annotating accurate, feature-rich data across color paintings, sketches, and text. Since we rely on original RGB images to draw and describe individuals, unclear ones hinder the creation of high-quality, appearance-consistent data—making it hard to fully and accurately capture people's appearance features.

• Many prior studies indicate that high-quality data enhances representation learning, and filtering unclear samples directly improves data quality. For instance, unlike CLIP, which trains on noisy samples, BLIP uses a data filtering mechanism to remove poorly matched image-text pairs—achieving better performance with fewer training samples.

We acknowledge that the aforementioned sample removal measures will make our high-quality benchmark somewhat fail to reflect real-world noises, which is discussed in our A.7 Limitation.

However, our cross-dataset generalization experiments in Table 7 validate the robustness of our proposed method, as most datasets here contain obvious noise (e.g., CUHK-PEDES has many rough, identity-mismatched text descriptions). The results show that training on our ORBench yields better generalization than other datasets, with our ReID5o method also exhibiting significant superiority. This indicates that ORBench and ReID5o complement each other to enhance robustness against real-world noise.

---

Q5: "In Table 1, those cross-modal methods also seem to perform well. Considering that these methods only support RGB and text, how did you train and test these methods on your dataset?"

A: Considering that RGB, infrared, color painting, and sketch data in the dataset essentially still belong to the visual modality, and these cross-modal methods all adopt a visual-text dual-branch architecture, we have adopted the following simple and direct approach to endow them with more multimodal retrieval capabilities:

• For RGB, infrared, painted, and sketched modalities, we directly use a visual encoder to extract features from each image sample. Most of these models use pre-trained CLIP as the encoder, and since CLIP has been pre-trained on hundreds of millions of image-text pairs, its visual encoder is not limited to the RGB modality. Therefore, this method is simple and feasible.

• In the training phase, similarly, we take RGB as the core of alignment and align the features of other modalities with RGB features to learn a unified multimodal space mapping. In the testing phase, for unimodal retrieval, it is only necessary to calculate the cosine similarity between each query and each RGB image in the gallery; for multimodal combined retrieval, we use the sum of the similarity ranking lists of each modal query as the combined similarity ranking list.

Through the above simple and direct method, we can train and test these cross-modal methods on our ORBench.

---

Q6: "Even though some visualized qualitative comparison examples, such as Figure 7, effectively demonstrate the high quality of ORBench compared to existing multi-modal datasets, could you conduct comparisons using quantitative metrics?"

A: Thank you for this question. A quantitative comparison can indeed better illustrate the high quality of our dataset.

Regarding the quantitative comparison of the text modality, in fact, in Figure 4, we verified the high quality of the text modality in our dataset by calculating and comparing the text information content across various datasets. The average Shannon entropy of the text data in our dataset is 5.53, which is higher than that of previous datasets. For example, CUHK-PEDES only has a Shannon entropy of 4.13.

Concerning the quantitative experiment on painting data, in Figure 5, we verified the high quality of the painting data in our dataset through manual evaluation, and it excels in both identity consistency and perspective conformity. The proportions of samples with identity consistency and perspective conformity scoring 3 or higher are 97% and 91.7% respectively.

Considering the current lack of effective automated evaluation methods for assessing the quality of painting data, we acknowledge that it is difficult for us to conduct a direct quantitative comparison of painting quality with other painting datasets at this stage. However, it can be clearly seen from Figure 7 that the painting quality of our dataset is significantly better than that of other datasets.

We guarantee that our dataset is of high quality, and we will make it publicly available to facilitate verification by the research community.

---

Last but not least, we would like to sincerely thank Reviewer akx3 again for the valuable time and constructive feedback provided during this review.

Thanks for your valuable comments of our work.

First, we would like to explain the research motivation and contributions of our work.

In real-world practical applications, when searching for a target person, the actual witness data used as queries is usually multi-source and multi-modal. Data from different modalities typically focus on different aspects of the target person's features. In practice, the number of modalities in such witness data is often uncertain, which raises a thought-provoking question:

For queries with an uncertain number of modalities, how can we facilitate feature complementarity between any different modal data to achieve an accurate portrayal of the target person and thereby improve the accuracy of the search?

To promote in-depth research on this issue, our work makes the following contributions:

1. We propose a new task with practical value and challenges, namely omni multi-modal person re-identification (ReID). This task requires achieving effective retrieval with varying multi-modal queries and their combinations in the ReID model, which remains largely unexplored in previous research.

2. We construct the first high-quality five-modal person ReID dataset named ORBench, where each person is accurately and comprehensively characterized by data from five modalities. Considering that existing datasets usually contain only a small number of modalities and are of poor quality, we have supplemented three additional modalities of data—namely painted, sketch, and text modalities—based on existing RGB-infrared datasets through extensive manual annotation and intervention. This dataset can serve as a valuable research platform to advance the development of the field.

3. We propose a simple and efficient multi-modal person ReID framework for retrieval with any combination of modalities named ReID5o, serving as a baseline method for the task and dataset. Through a multi-expert routing and multi-modal feature fusion mechanism, this method can effectively achieve feature information complementarity between any modalities, thus obtaining more accurate retrieval results.

We believe that the task, dataset, and method, as an integrated whole, complement each other. They simulate real-world practical application scenarios and can provide a foundation and inspiration for subsequent research on multi-modal person ReID.

---

Now, we would like to address your specific concerns through the following detailed responses.

Q1: "Most modalities in the dataset introduced by this paper are artificially generated rather than captured from real-world sources."

A: We acknowledge that some modalities in our dataset are artificially generated, but we have taken rigorous measures to ensure all modalities closely align with real-world scenarios and capture their inherent complexity.

• For RGB and infrared modalities: The data in ORBench are filtered from two existing public datasets, LLCM and SYSU-MM01. These datasets contain RGB and infrared images directly collected from real-world scenes, with diverse distributions in illumination, weather, and scenarios, ensuring the pedestrian appearance features are real and representative of real-world variations.

• For the text modality: The descriptions are manually annotated by 26 human annotators following strict guidelines, ensuring rich language styles, accurate reflection of real pedestrian attributes, and consistency with real-world semantic expression.

• For sketch and color pencil modalities: These data are AI-generated with heavy human intervention, not via repeated RGB transformations. Instead, they mimic real creation: AI produces multi-view sketches and drawings by referencing real RGB images (appearance) and detailed text (semantics). Humans refine outputs to match real sketch/pencil traits (e.g., preserving key features, simplifying non-essentials, matching textures) rather than replicating RGB content. This labor-intensive process captures real-world sketch/pencil perception complexity.

Our experiments confirm the dataset's effectiveness and generalization. As shown in Figure 6, the model trained on our dataset leverages multimodal complementarity to achieve accurate retrieval, demonstrating adaptability to real-world multimodal variations. Additionally, Table 7's dataset generalization experiments show our dataset outperforms others, indicating our dataset helps to learn robust representations transferable to real-world complexity.

---

Q2: "Most of the baseline methods are re-implemented and fine-tuned on this dataset by the authors themselves. The paper lacks cross-dataset transfer evaluations on established benchmarks."

A: Regarding the re-implementation and fine-tuning of baseline methods, it is worth noting that this study focuses on the novel task of person ReID across arbitrary modalities and their combinations, with ORBench being the first five-modality dataset for it. Since both are new, existing baselines lack training/evaluation under such multimodal settings and have no reusable weights. Therefore, to verify the effectiveness of ReID5o within a unified experimental framework, we re-implement existing methods and perform fine-tuning on ORBench. This is a necessary step in verifying the effectiveness of new methods in pioneering work in this field, ensuring fair and relevant comparisons.

Regarding cross-dataset transfer evaluation, we have actually conducted relevant experiments specifically in the paper (see Table 7 for details). We transferred the ReID5o model and baseline methods trained on ORBench to multiple existing cross-modal benchmarks for evaluation. The results clearly show that ReID5o still maintains significant advantages in cross-dataset scenarios, and the models trained on the ORBench dataset significantly outperform those trained on other datasets in terms of transfer performance. This result not only demonstrates the generalization ability of ReID5o but also verifies the rationality of the ORBench dataset, ruling out the impact of dataset design biases on performance.

---

Q3: "The paper lacks a thorough discussion of the trade-offs between efficiency and performance during training and inference, as well as the model's scalability under constrained hardware resources."

A: Regarding the trade-off between model efficiency and performance, we have actually conducted a systematic consideration in our research and verified it through experiments. The specific explanations are as follows:

In fact, in terms of the trade-off between efficiency and performance, we specifically compared the impact of different multi-expert routing designs on model performance and computational overhead in Table 3.

The experimental results show that the selected trade-off parameters not only achieve the best performance improvement but also only bring a small increase in params and FLOPs (please refer to Table 3 for specific values and experimental explanations). This result directly reflects the importance we attach to the balance between efficiency and performance in the design.

From the perspective of model architecture design, the efficiency of ReID5o is mainly reflected in three aspects:

• The shared transformer encoder significantly reduces redundant computations in multi-modal scenarios and avoids parameter explosion caused by designing separate encoders for different modalities;

• The lightweight tokenizer controls additional overhead while ensuring modal adaptability by simplifying the modal feature mapping process;

• The LoRA-based multi-expert routing mechanism realizes parameter-efficient fine-tuning through low-rank matrix factorization, which significantly reduces the computational complexity of the routing module compared with traditional multi-expert models.

The above designs enable the model to maintain high performance while having good hardware adaptability, and it can still run in environments with limited computing resources.

---

Q4: "To construct multimodal query combinations, the authors randomly sample data from different modalities but of the same identity, resulting in overly consistent intra-identity pairs. The model may exploit superficial shortcuts instead of truly demonstrating cross-modal generalization capabilities."

A: First, our task focuses on omni multi-modal person ReID, whose core goal is accurate retrieval via any modality combinations. This task inherently requires multi-modal query combinations to maintain identity consistency—a necessity, not an artificial constraint. The aim is to enable the model to leverage complementary information across modalities, constructing more comprehensive pedestrian representations for real-world cross-modal search. This aligns with practical scenarios where users query based on diverse modal data of the same target.

Second, our experimental design rigorously verifies cross-modal generalization. We separated the dataset into 600 training and 400 test identities, ensuring independent division to evaluate performance on unseen data. Additionally, cross-dataset experiments in Table 7 explicitly demonstrate the model learns essential inter-modal associations (not dataset-specific superficial features), confirming strong cross-scenario generalization.

Finally, significant modal differences in our dataset prevent over-reliance on superficial shortcuts. Sketch and color painting modalities include multi-perspective samples with visual characteristics distinct from RGB images; text descriptions use varied styles to ensure richness and diversity. These designs force the model to learn intrinsic inter-modal connections, instead of relying on superficial similar features, thus effectively improving the model's cross-modal generalization ability.

---

We would like to sincerely thank Reviewer AoA7 again for the valuable time and feedback provided.

Thanks for your valuable comments. Now we respond to your concerns as follows.

---

Methodological details are unclear

Q1: "How are the features exactly fused?"

A: Thanks for identifying an ambiguity.

To clarify, the combinatorial traversal is performed not on individual modal features $Z^{mod}$ but on the entire set of multimodal query features $\{Z^I, Z^C, Z^S, Z^T\}$. Specifically, we generate all possible combinations from single-modal to quad-modal (i.e., $C_4^1=4$, $C_4^2=6$, $C_4^3=4$, and $C_4^4=1$ combinations). These are then concatenated and processed by the Feature Mixture module.

The example in Lines 221-225 (IR+Text fusion) illustrates this process for one dual-modal combination. We will revise the text to resolve this confusion.

Q2: "Detailed explanations on task setup, retrieval procedure and feature size."

A: The task is essentially a fine-grained multimodal image retrieval task that generalizes to unseen identities (not included in the training set). This is a metric learning task rather than an identity classification task.

In training, we follow common practice by using training-set identity labels to guide the representation learning, with two losses:

• The primary loss is the similarity distribution matching (SDM) loss, a typical metric learning loss that aligns cross-modal features by minimizing the KL divergence between cross-modal similarity distributions and normalized label-matching distributions.

• A supplementary identity classification loss serves as an auxiliary objective to enhance feature discriminability, using identity labels for multi-class classification of each multimodal sample.

In inference, the classification layer is entirely discarded. Retrieval is performed by computing cosine similarity between the query feature and all gallery features.

For the feature size, we use CLIP-B/16 as the encoder (consistent with prior methods) with its default size of 512.

As suggested, these details will be explicitly stated in the revised version.

---

Unclear how laborious creating the dataset was

It took months of intensive effort to ensure quality and diversity of ORBench—no mere crude combination/extension of existing resources. Key workflow details:

• Manual filtering of existing infrared-RGB datasets. Some datasets (notably LLCM) contain noisy samples with extremely poor imaging quality, where facial features are barely distinguishable to the naked eye—unsuitable for accurate annotation and lacking sufficient learnable features. Initial attempts at automatic filtering via resolution thresholds failed, as resolution does not strictly correlate with facial clarity. Thus, we adopted manual screening: for each identity, all images were reviewed individually, and low-quality samples were removed.

• Manual fine-grained text annotation for RGB images. We developed strict annotation guidelines specifying requirements for objective, unbiased and explicit appearance descriptions, followed by training annotators (including trial sessions) to ensure compliance. The task was completed by 26 annotators over approximately one month. Our team then rigorously reviewed all annotations, with inappropriate descriptions returned for revision.

• AI-assisted generation of paintings and sketches with human oversight. AI assistance was chosen for two reasons: manual drawing demands high expertise, and large-scale manual creation is time-prohibitive. After testing multiple tools, we selected Doubao's artistic painting tool. The workflow involved: manually selecting the clearest RGB images per viewpoint; adding detailed manual text descriptions; feeding these to the AI to generate color paintings; and having human reviewers inspect outputs (with substandard ones recreated). Sketches were derived via style conversion from these paintings. This hybrid approach ensured efficient production of high-quality data.

Q3: "Did annotators use tools, and did this introduce biases or repeated phrasing?"

A: Annotators strictly followed guidelines for manual annotation without tools. Specialized reviewers mitigated potential biases. While some repetition is inevitable, the 26-annotator team enhanced linguistic diversity, reducing redundancy.

Q4: "Might later descriptions be lower quality than earlier ones?"

A: Specialized reviewers manually evaluated all annotations, with substandard entries returned for revision—effectively preventing quality degradation.

Q5: "Why use AI for drawings but not LLMs for text?"

A: Text descriptions require detailed references to RGB images, which pure LLMs cannot process. MLLMs, while capable of image description, struggle with low-resolution person images and fine-grained details, failing to generate satisfactory results. Thus, we opted for manual annotation for text. For drawings, AI (with human oversight) efficiently produced high-quality outputs, making it the practical choice.

---

How realistic is the dataset, and have meaningful choices been made?

Q6: "What real-world setting(s) are exactly being simulated?"

A: Our five-modal dataset and task setup align with practical needs, explicitly simulating real-world scenarios. A key example is public safety: when searching for a target, investigators often use multi-modal information—such as low-resolution surveillance footage, witness descriptions, and sketches/portraits derived from these descriptions. Effectively leveraging their complementarity to characterize targets for accurate identification remains a highly practical research topic. Additionally, real-world scenarios naturally involve variable numbers of modalities (from single to multiple). Our work addresses retrieval under arbitrary modal combinations, enhancing practical relevance.

Q7: "Were these AI-generated annotations added just to look good, or do such sketches reflect intended real-world uses?"

A: These modalities were included based on practical value rather than superficial visual appeal. In real scenarios, after witnesses provide textual descriptions, more precise drawings or sketches are often created to improve search accuracy, with subsequent retrieval combining all available modalities. Our dataset faithfully reproduces such practical cases.

Q8: "In intended use cases, will all modalities have the same detail and completeness? What variance can we expect within and across them?"

A: In intended use cases, variations in detail and completeness across modalities are inherent—this is a core challenge in real-world multimodal ReID, which our dataset deliberately incorporates.

Regarding intra-modal variance: RGB/infrared data exhibit fluctuations in color, contour, and clarity due to lighting and scene conditions; Sketches/paintings show variability from AI generation randomness and multi-perspective design; Texts vary in descriptive content due to stylistic differences among 26 annotators.

Inter-modal differences appear in information type and granularity (Figure 7): visual modalities focus on spatial features (RGB with color, infrared with temperature, paintings more abstract), while texts are semantic-symbolic with lower detail granularity.

These differences reflect real-world realities and highlight the challenges our dataset presents to practical algorithms.

---

Ethical concerns

Q9: "Privacy of people in the benchmark"

A: We clarify that both the original benchmarks and our extended data strictly adhere to the guidlines with explicit privacy safeguards.

For SYSU-MM01, we verified its creators obtained written privacy licenses from all captured pedestrians, explicitly permitting use of their images for scientific research and academic publication.

For LLCM, its creators used MTCNN for facial blurring to anonymize personal identifiers. We confirmed this meets standard anonymization criteria, aligning with NeurIPS' privacy risk minimization requirements.

Both datasets have usage agreements with privacy protection provisions; access requires strict adherence and formal signing. Additionally, we obtained written permission from both datasets' creators to modify and extend them.

Q10: "Bias and discrimination"

A: Our data comes from public datasets complying with local laws, with measures ensuring fairness and avoiding bias or discrimination. Sourced from Sun Yat-sen and Xiamen Universities, it covers their campus communities, with balanced age and gender distribution. The datasets contain no biases or specific protected groups and comply with NeurIPS guidelines. Our human text annotations, under strict review, are objective and unbiased, unable to identify protected categories. Moreover, due to rigorous data review and compliance, our work will not retrieve individuals by ethnicity, age, or disability.

Q11: "Answers in questions in the NeurIPs checklist do not align with what can be read in the paper submission"

A: For Question 12: We communicated in writing with dataset creators, obtained permission for data use and extension, and strictly complied with their licenses. Additionally, we've specified the research license (CC BY-NC-SA 4.0) on OpenReview. We will also follow your suggestion to explicitly note the licenses in the supplement.

For Question 13: Since we did not directly submit the asset (e.g., via an anonymized URL) for this submission, we marked [N/A]. For further details, please see earlier statements.

For Question 14&15: We initially understood these two questions to refer to large-scale, direct human experiments (e.g., distributing mass questionnaires). As our research is unrelated, we marked [N/A]. We confirm strict adherence to relevant guidelines and that all workers were paid adequately. For further details, please see earlier statements.

---

We would like to sincerely thank Reviewer h3Ed again for the valuable time and feedback.

---

We will explicitly state in the revised manuscript that our dataset does not fully capture the variance and imperfections of real-world scenarios.

To be perfectly clear, we agree with the reviewer's conclusion. We will explicitly state in the revised manuscript that our dataset does not fully capture the variance and imperfections of real-world scenarios. Specifically, we will:

1. Revise the dataset creation section to explicitly state the idealized assumption and introduce the "viewed sketch" vs. "forensic sketch" dichotomy, citing [Klare'10]. We will clarify that the identity consistency across modalities in our dataset is intentionally high, and our query modalities are, in essence, high-quality "viewed sketches" and their equivalents, not "forensic sketches."

2. Add a dedicated paragraph in the Limitations section to discuss the points raised above, clearly delineating what our dataset does and does not achieve concerning real-world forensic applications. We will acknowledge that the challenge of matching low-quality, ambiguous, or partially inaccurate "forensic sketches" is a critical but different problem that our dataset does not directly address.

---

We concur with the reviewer's positive framing that our work "presents an okay starting point." This was precisely our intention. By providing the community with a large-scale, high-quality, multi-modal benchmark, we aimed to establish a foundation. We believe our dataset can catalyze research in several ways: as a powerful pre-training resource, as a testbed for developing ideal feature alignment algorithms, and as a baseline from which future research can explicitly tackle the domain gap between "viewed" and "forensic" data. We hope our work encourages future efforts to build upon this by introducing more realistic noise and variations.

Once again, we are grateful for the reviewer's detailed and constructive feedback, which will undoubtedly strengthen our paper. We hope our response and proposed revisions have adequately addressed their concerns.

Best regards,

The Authors of Submission 5021