Instance-level Consistent Graph With Unsupervised Human Parts for Person Re-identification

We sincerely appreciate your constructive and thoughtful feedback. Below are our supplementary experiments and responses:

1. Weakness 1

Thank you for your valuable suggestion.

While several modules in our framework rely on established methodologies, the proposed approach of leveraging a more flexible instance-level consistency for person re-identification, instead of pursuing strict part-level consistency, proves to be practical and effective.

Our ICG framework utilized attention-based foreground mask to extract pedestrian masks, ensuring that subsequent operations are performed on enhanced foreground features. This approach effectively mitigates the issues of fine-grained human part misalignment and coarse-grained image block misalignment outlined in Figure 1 of our paper. Additionally, pixel-wise human parts clustering enables unsupervised clustering of pedestrian parts. In the PPC module, features from all images of the same ID are clustered together, ensuring that the model maintains instance-level consistency during the clustering process. Furthermore, the flexible structure graph constructs a dynamic graph for each query image. This graph uses the human parts and their labels obtained via K-means clustering as the initial graph matrix, which is then updated through multiple GCN layers. The GCN strengthens the relationships between correlated human parts and attenuates the effects of misalignment caused by partial occlusion, pose variations, and extreme illumination, thereby forming more robust pedestrian structural features.

We have expanded the discussion in appendix to highlight potential applications of ICG in real-world scenarios (e.g., smart city surveillance, public transportation safety). These use cases illustrate the practical impact and versatility of our approach.

In addition, inspired by the reviewer fRrV, based on our flexible instance-level consistency framework ICG, incorporating advanced clustering techniques, modern self-attention mechanisms, or graph learning approaches may further enhance the methodology.

2. Weakness 2

We added an ablation experiment about the number of clustering centers $K$ in PPC module, and conducted further analysis on the impact of the number of clusters K on the generation of local semantic regions by visualization figure. The detailed explanation could be find in our response to fRrV (PART II) 3.Weakness 3

3. Weakness 3

We sincerely thank you for pointing the potential limitations of relying on established methodologies. The motivation of our proposed ICG framework, as well as the realization ideas for its components, has been discussed in our response to weakness 1. The ICG framework introduces a more flexible instance-level consistency approach, and our experiments have demonstrated the feasibility and effectiveness of this framework.

4. Weakness 4

Thank you for your insightful suggestion regarding the importance of including up-to-date results to prove the effectiveness of our method. We removed certain outdated methods from our comparison updated some recent methods. Specific modifications could be referred to response to Reviewer xMUt Weakness 2.

Regarding the performance on the Market-1501 dataset, we would like to highlight two important observations:

Performance Saturation: Many recent methods achieve mAP scores of 91-92 on Market-1501, indicating near-saturation. This limits the dataset’s ability to distinguish newer methods or reflect advancements on more complex datasets.

Re-Ranking Impact: Many methods use re-ranking to boost mAP by 1-2 points, which can obscure the model’s true discriminative ability. To ensure fair comparison, we avoided re-ranking in our evaluation.

5. Weakness 5

Due to space limitations, the original manuscript lacked sufficient detail regarding the similarity measurement between query and test images, as well as the loss function. Here, we provide a brief explanation and will include more detailed descriptions in the appendix of the revised version, covering the output features of each module and the loss function setup. This will allow readers to better understand and replicate the proposed method.

• Similarity measure

During the testing phase, a total of four types of features can be obtained for each image to be queried: global feature ${F_{global}}$, foreground feature ${F_{foreground}}$, semantic feature ${F_{part}}$, and graph features $F_{graph}$. The proposed network in this paper employs cosine distance between features to measure the similarity between the query image and the image to be queried.

Since the number of part-level semantic features is adaptive and some part-level semantic areas are invisible when occluded, only visible part-level semantic features are selected for distance calculation. The total feature distance is divided into two parts: fixed global features, foreground features and graph features, and variable number of part-level semantic features.

Let $D(\cdot \text{ , }\cdot )$ denote the cosine distance between features, the distances for global features, foreground features, graph features, and the $k$th semantic features of the query image $q$ and the image $g$ are: ${d_{global}}=D(F_{global}^{q},F_{global}^{g})$, ${d_{foreground}}=D(F_{foreground}^{q},F_{foreground}^{g})$, ${d_{graph}}=D(F_{graph}^{q},F_{graph}^{g})$, ${d_{part-k}}=D( F_{part-k}^{q},F_{part-k}^{g})$.

If the $k$th part-level semantic feature of the query image q and the query image g both exist, then $l_k^q⋅l_k^g=1$. In other cases, $l_k^q⋅l_k^g=0$. The final average similarity distance is:

$

d=(\sum\limits_{k=1}^{K-1}{e_{k}}{d_{part-k}}+( {d_{global}}+{d_{foreground}}+{d_{graph}}))/(\sum\limits_{k=1}^{K-1}{l_k^q⋅l_k^g}+3)

$

• Loss function

In the training phase, the loss function of the network consists of three components: loss of AFM, PPC, and composed features.

The loss of AFM is the cross-entropy loss constituted by the foreground confidence map ${P_f}(x,y)$ and the foreground mask:

$

{L_{sep}}=\sum\limits_{x,y}{-}\log{P_{f_i}}(x,y)

$

The number of channels was changed to $K$ dimensions based on the foreground feature map ${M_g}$, using a $1{\times}1$ convolution kernel as a linear layer, as described above, for each pixel site prediction. With the softmax classifier, $K$ confidence maps are obtained, which are expressed as ${{P}_{k}}(x,y)$

The $K$-dim vector composed of $P(x,y)$ at the spatial location $(x,y)$ is optimized with the pseudo-label ${{k}_{i}}$ obtained from clustering using cross-entropy loss to obtain the loss of PPC as:

$

{{L}_{par}}=\sum\limits_{x,y}{-}\log {P_{k_i}}(x,y)

$

On the other hand, the features were processed through the BNNeck module following the optimization of BoT. The composed feature $F_c$ for retrieval includes global features ${F_g}$, foreground features ${F_f}$, part features ${F_p}$, and structural features ${F_s}$. Each type of features corresponds to a loss group $L$ consists of the triplet loss, center loss, ID classification loss with label smoothing. Let $\mathbb{L} = { L_g, L_f, L_p, L_s }$ is the set of losses for four types of features, the loss of the composed feature can be presented as:

$

L = L_{ID} + L_{tri} + L_{cen}, L \in \mathbb{L} 

$

$

L_{feat} = L_g + L_f + L_p + L_s % \sum_{L{\in}\mathbb{L}}{L}

$

The overall objective function is:

$

L_{opt} = \alpha_{feat}L_{feat} + \alpha_{sep}L_{sep} + \alpha_{par} L_{par}

$

Where each $\alpha$ is a balance weight, the experimental settings are 0.2, 0.1, and 0.1 for $\alpha_{feat}, \alpha_{sep}, \alpha_{par}$, respectively.

We sincerely appreciate your constructive and thoughtful feedback. Below are our supplementary experiments and responses:

1. Weakness 1

• Testing on additional datasets

To address your concerns about generalizability, we conducted experiments on the CUHK03 dataset, another classic pedestrian re-identification dataset. We presented the performance comparison between the proposed ICG framework and state-of-the-art methods on MSMT17 dataset and CUHK03 dataset. Most of the experimental results are derived from the literature and are summarized in the table below.

In conclusion, the ICG framework achieves the best performance on the MSMT17 dataset and performs second only to AAformer on the CUHK03 dataset, demonstrating the effectiveness and Generalizability of the proposed approach.

• Testing in real-world scenarios

Due to time constraints, we were unable to collect and annotate a large-scale dataset from real-world scenarios. However, we tested the proposed ICG framework in practical surveillance scenarios. Specifically, we evaluated its performance under three indoor and outdoor cameras on the same pedestrian with varying viewpoints. The ICG framework successfully re-identified the same individual across all settings. While we cannot include test images in this response, we will provide the example in the appendix of the revised paper.

• Model complexity analysis

The follow table presents a comparison of the computational complexity of the proposed ICG framework with other leading algorithms in terms of model size, floating-point operations, and performance on Market-1501 Dataset. While TransReID achieves performance comparable to our proposed algorithm, its model is more complex. Similarly, MFA incorporates motion information at the feature map level, resulting in higher model complexity and computational cost. In contrast, the ICG framework introduces three simple yet effective modules, achieving superior performance with reduced complexity.

2. Weakness 2

The appearance changes in pedestrians' clothing can affect re-identification performance. The ICG framework, based on instance consistency, segments human body parts using a clustering algorithm. If a pedestrian’s clothing changes, the part-level features may differ significantly, which could hinder matching.

The accuracy of the foreground mask also plays a crucial role. Our attention-based mask, which avoids the need for an additional semantic parsing model, suffers from blurred edges, potentially including background information that affects performance.

Additionally, pedestrian re-identification methods based on Transformers and infrared-visible fusion have advanced rapidly in recent years. While the ICG framework, relying on CNNs for part segmentation, shows strong results, we recognize the importance of exploring and integrating emerging techniques in future work.

We sincerely appreciate your constructive and thoughtful feedback. Below are our supplementary experiments and responses:

1. Weakness 1

Thank you for your valuable suggestion.

Recent advancements leveraging human parts extraction have shown promise by utilizing discriminative part features. However, the detailed information provided by human parts also presents challenges, particularly in terms of part misalignment. Real-world re-ID scenarios often face issues such as partial occlusion, pose variations, and extreme illumination, which exacerbate misalignment due to limited information about human parts. Despite previous efforts, achieving part-level consistency across all samples remains an ideal but stringent assumption, especially under challenging conditions. Extracting identical features from incomplete information about human parts can lead to a semantic gap between samples of the same instance.

Our ICG framework addresses these challenges by utilizing an attention-based foreground mask to extract pedestrian masks, ensuring that subsequent operations are performed on enhanced foreground features. This approach effectively mitigates the issues of fine-grained human part misalignment and coarse-grained image block misalignment outlined in Figure 1 of our paper. Additionally, pixel-wise human parts clustering enables unsupervised clustering of pedestrian parts. In the PPC module, features from all images of the same ID are clustered together, ensuring that the model maintains instance-level consistency during the clustering process.

Furthermore, the flexible structure graph constructs a dynamic graph for each query image. This graph uses the human parts and their labels obtained via K-means clustering as the initial graph matrix, which is then updated through multiple GCN layers. The GCN strengthens the relationships between correlated human parts and attenuates the effects of misalignment caused by partial occlusion, pose variations, and extreme illumination, thereby forming more robust pedestrian structural features.

While several modules in our framework rely on established methodologies, the proposed approach of leveraging a more flexible instance-level consistency for person re-identification, instead of pursuing strict part-level consistency, proves to be practical and effective. We will include additional experiments and ablation studies to further demonstrate how each component contributes to the overall performance.

2. Weakness 2

We sincerely thank the reviewer for pointing out the potential limitations of using attention-based foreground mask, basic K-means clustering and a standard graph convolutional network (GCN) architecture in our work. The motivation of our proposed ICG framework, as well as the realization ideas for its components, has been discussed in our response to weakness 1. The ICG framework introduces a more flexible instance-level consistency approach, and our experiments have demonstrated the feasibility and effectiveness of this framework.

However, we acknowledge that incorporating advanced clustering techniques (e.g., spectral clustering, deep embedded clustering), modern self-attention mechanisms, or graph learning approaches may further enhance the methodology. But due to time constraints, we are unable to implement and evaluate these approaches in the current work.

We deeply appreciate the reviewer’s constructive feedback and insightful suggestions. In future work, we plan to explore transformer-based structures and and develop end-to-end instance-consistent re-identification algorithms. Our further research will aim to leverage transformer-based attention mechanism's global dependency, investigate more precise pedestrian part segmentation, and construct refined pedestrian structural graphs.

4. Question 1

Thank you for your valuable suggestions. The comprehensive ablation studies could be find in the above response to Weakness 3. The ICG's behavior and limitations could be find in find in response to Reviewer 7m9c (PART I) Weakness 2. The computational complexity analysis could be find in response to Reviewer 7m9c (PART I) Weakness 1 third point.

5. Question 2

• Comparisons with recent transformer-based approaches

As shown in the below Table, our proposed ICG framework achieves competitive or superior results compared to recent transformer-based approaches. ICG attains the best performance with a Rank-1 accuracy and mAP on Market-1501, DukeMTMC-reID and MSMT17 dataseta, but ICG achieves a Rank-1 accuracy of 76.9% and mAP of 77.4% on CUHK03 dataset, getting a lower result compared with AAformer. AAformer introduces an auto-aligned transformer that automatically locates both human and non-human parts at the patch level. Its self-attention mechanism models global dependencies and ensures fine-grained patch alignment. This may be the reason why ICG's performance is slightly worse than AAformer on some datasets.

• Qualitative results showing the clustering and graph construction process

Some of the qualitative results are mentioned in the above response. Furthermore, in order to explore the variation of pseudo-labels in local semantic regions with training, we visualized the pseudo-label change process of each local semantic region when K is 6. More figures and explanations of the experimental results can be referred to the appendix of revised paper that we will upload later.

3. Weakness 3, Question 1

Thank you to your suggestions. Due to space constraints, we mistakenly omitted some ablation studies. The detailed ablation studies is added in Section 3.3 ablation experiments as follow. The computational complexity analysis could be find in response to Reviewer 7m9c (PART I) Weakness 1 third point.

(1) We further analyse of the AFM performance in the three scenarios shown in Figure 7:

“The effectiveness of the AFM is verified across various scenarios. Furthermore, specific foreground areas exhibit stronger responses, indicating their heightened importance. These areas contain information that better discriminates inter-class differences, thereby facilitating classification tasks. Additionally, the AFM effectively suppresses responses in occluded regions. To a certain extent, it mitigates the impact of background noise, resulting in less noise in human parts.”

(2) We will add an ablation experiment about the number of clustering centers $K$ in PPC with the following description:

Intuitively, the number of cluster centers, denoted as $K$, determines the granularity of aligned parts when generating part pseudo-labels. In this approach, multiple part regions are obtained by clustering from top to bottom. The larger the value of $K$, the smaller the pixel share of each region, resulting in finer granularity. Consequently, the PPC generates different numbers of confidence maps for the classification of pixel channel features.

In order to explore the influence of the number of clustering centers K on the network performance, the ablation experiment is conducted as the follow table:

From the results presented in the table, it can be observed that the experiments achieve near-optimal performance at K=6. To approximate real-life scenarios, images often include personal belongings such as backpacks. When the number of clusters is set to K=4, the generated local semantic regions may be relatively accurate, leading to a local optimum. However, when the number of clusters increases to K=7, the granularity of the generated regions becomes too fine, resulting in less effective local features for pedestrians and ultimately degrading network performance. At K=6, personal belongings are identified with the highest probability.

Additionally, we conducted further analysis on the impact of the number of clusters K on the generation of local semantic regions by visualization figure. The visualizations of these experiments are provided in the appendix, comparing the effects of clustering algorithms with various K values against an additional semantic parsing model. As shown, at K=6, potential personal belongings are effectively identified and incorporated into the pedestrian representation. Figure (b) illustrates the parsing results under occlusion conditions. The first two rows show that the PPC module achieves performance nearly comparable to the SCHP algorithm, effectively distinguishing occluded areas. The results in the last row highlight the advantages of our approach in handling occlusion caused by other pedestrians. SCHP struggles to distinguish such cases, treating features of other pedestrians as interference. In contrast, our method clusters features from all images of the same ID, allowing information sharing across instances. This enables our model to discard features of other pedestrians as background, demonstrating the benefits of our approach. Moreover, observing each column in the figure, regardless of the value of K, the PPC module consistently ensures local semantic region alignment across images.

We sincerely appreciate your constructive and thoughtful feedback. Below are our responses and detailed explanation:

1. Weakness 1

The proposed attention-based foreground mask (AFM) learning method differs from approaches that rely on additional semantic information, such as skeleton poses, human part segmentation, or bounding boxes. These methods typically involve pre-trained detection or parsing models, which increase model complexity. It also contrasts with coarse partitioning methods for pedestrian images, which often lead to part misalignment issues.

In prior work [1], researchers utilized RGB-Mask pairs as inputs to learn features from the body and background regions separately for contrastive learning. This method first employs a pre-trained pedestrian segmentation model to generate the pedestrian mask, which is then combined with the RGB image as input. Each image is augmented with the mask as prior information, and pedestrian re-identification features are learned through three main streams (the full-stream, the body-stream, and the background-stream).

Our method differs with [1] in several key aspects. First, we rely solely on the original dataset for training, with no need for additional data as model input. Second, the AFM module is based on the observation that the foreground response in feature maps tends to be larger than the background response. The spatial attention layer enhances the contrast between the foreground and background by increasing the attention values of foreground pixels. Through classification loss and part-parsing loss, the network is progressively guided to focus more on foreground features during learning. Moreover, AFM selectively enhances the foreground and suppresses the background at feature-level. In contrast, features extracted from image-level foreground mask (by [1]) may include noise at the mask edges due to transitional areas. Our method effectively avoids this limitation, improving the robustness of the learned features.

2. Weakness 2

Thank you for your valuable suggestion. We removed certain outdated methods from our comparison updated some recent methods. A more complete comparison table will be given in the revised version of the paper. As shown in the below Table, our proposed ICG framework achieves competitive or superior results compared to these recent approaches. ICG attains the best performance with a Rank-1 accuracy and mAP on DukeMTMC-reID and MSMT17 dataset, and ICG achieves a Rank-1 accuracy of 95.4% and mAP of 88.9% on Market-1501, matching or surpassing AAformer and MSINET.

MSINet employs a multi-scale interaction search strategy, significantly enhancing the model's discriminative power through contrastive learning of objects. Our ICG framework did not use multi-scale feature extraction technology, which may lead to the loss of some details captured by the network. AAformer introduces an auto-aligned transformer that automatically locates both human and non-human parts at the patch level. Its self-attention mechanism models global dependencies and ensures fine-grained patch alignment. This may be the reason why ICG's performance is slightly worse than AAformer on some datasets.