Visually Consistent Hierarchical Image Classification

3. Limited Evaluation on ImageNet

Limited evaluation. ImageNet, as a large-scale hierarchical dataset, might provide a stronger test of your method's capabilities compared to the smaller datasets currently used.

We appreciate the reviewer’s comments and would like to clarify our dataset choices and experimental setup.

First, CUB and Aircraft are the most commonly used benchmark datasets in prior studies [1, 2, 3, 4], and we conducted our experiments on these datasets to ensure comparability with existing methods. Additionally, to evaluate our approach on a larger and more challenging subset that includes diverse classes beyond a single type (e.g., birds or aircraft), we conducted experiments on BREEDS, a subset of ImageNet.

Regarding ImageNet-1K, its hierarchy is highly imbalanced, making it challenging to apply our approach. As a result, most hierarchical classification studies on this dataset have focused on flat-level classification, not multi-granularity classification.

We are currently conducting additional experiments on large-scale iNaturalist dataset to further validate our method on a larger dataset. We will update the results as they become available. However, we kindly ask for your understanding, as our training environment relies on a single GPU (Nvidia A40), which causes the experiments to take longer to complete.

[1] Your “Flamingo” is My “Bird”: Fine-Grained, or Not, 2021
[2] Consistency-aware feature learning for hierarchical fine-grained visual classification, 2023
[3] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024
[4] HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification, 2024

---

4. Formatting problem

Formatting problem: Please ensure the readability of all components of the paper. For instance, the font size in the tables is small, which may make it difficult for readers to check the data presented

Thank you for pointing out the formatting issue. Due to the page limit, we had to make certain adjustments to fit the content. However, we revised Table 2 to increase its font size for better readability. If there are specific tables or components that are still difficult to view, please let us know, and we will address them further.

---

We hope this clarifies the reviewer’s concerns. If there are any further concerns or clarifications needed, we would be happy to discuss them further.

Dear reviewer uxN6,
Thank you for your valuable feedback and comments. We address your concerns and questions in the response below.

1. Overlap with TransHP (NeurIPS 2023)

1. Overlap with former work? I noticed that an important reference, TransHP: Image Classification with Hierarchical Prompting (NeurIPS 2023), is not cited in your paper. ....

We would like to clarify the differences between our work and TransHP to address the concern. While both works use ViT backbones, our approach and TransHP differ fundamentally in both goals and methodology:

1. Goal:
   TransHP focuses on improving fine-grained classification by leveraging hierarchical information as auxiliary supervision. In contrast, our work targets multi-granularity classification, aiming to make predictions across all levels of the hierarchy simultaneously while ensuring semantic consistency between levels.

2. Method:
   The approach to coarse-level supervision is also significantly different. TransHP uses learnable prompts, with one prompt per coarse label, to guide fine-grained predictions. On the other hand, we directly apply coarse-level supervision to the output class tokens of each block, enabling independent predictions at different levels and facilitating consistent visual grounding across the hierarchy.

These differences demonstrate that our work addresses a distinct challenge with a fundamentally different methodology. We included TransHP in the related work of the revised manuscript and clarify these distinctions to avoid any potential confusion.

---

2. Limited novelty

Limited novelty. Your proposed approach introduces elements such as Superpixel and Graph pooling. While these are effective, both are well-established techniques in computer vision....

The use of superpixels and graph pooling is not part of our proposed contributions. As stated in L183-185 of the original manuscript, these are components introduced by CAST, and we clearly attribute them as CAST's innovations.

Our novel contributions are as follows:

(1) Key Insight:
Our observation revealed that classification at different granularities involves fundamentally distinct tasks requiring attention to different but consistent regions within an image. We found that inconsistencies arise because each classifier tends to independently attend to different regions without connection. This observation led us to propose consistent visual grounding as a novel solution to connect hierarchical classifiers across levels.

(2) Leveraging Semantic Segments for Hierarchical Classification:
While we adopted CAST as part of our architecture, it is important to emphasize that CAST originates from a different task, weakly-supervised semantic segmentation. In CAST, “hierarchy” refers to “part-to-whole” visual grouping (e.g., eyes, nose, arms), while our work addresses a “taxonomy hierarchy” (e.g., bird - Green Hermit). It was NOT evident that the concept of “part-to-whole” segments would align well with a taxonomy hierarchy; this connection is a novel discovery introduced through our work.

In addition, based on our observation in (1), we newly propose leveraging segments at different granularities to enhance multi-granularity classification. To the best of our knowledge, the use of segments has NOT been applied to hierarchical classification tasks.

Thus, this adaptation is neither trivial nor an obvious solution; it stems from our novel observation and bridges two distinct fields to tackle challenges unique to hierarchical classification.

We hope this summary highlights the novelty and importance of our work.

We appreciate the reviewer’s feedback, and we would like to provide further clarification.

1. The difference between TransHP

(1) TransHP’s variant in Fig.4 (2) is NOT "exactly the same" as ours. Instead, it can be regarded as the Hier-ViT baseline, which is among the baselines we employed for comparison.

(2) While both our method and TransHP utilize a ViT backbone and incorporate coarse labels, the similarities are limited to these general design choices. The fundamental differences lie in the motivation, the role of coarse labels, and the methodology to hierarchical classification.

To clarify, hierarchical classification poses a unique challenge: while the input image remains constant, the output shifts across the semantic (text) space (e.g., from "bird" to "Green Hermit"). Due to this formulation, existing works, including TransHP, primarily address this challenge by embedding data into the semantic space.

Specifically, TransHP employs coarse labels as prompts to guide fine-grained classification within the hierarchy, operating primarily in the semantic space. The focus is on refining predictions within the scope of coarse label prompts, embedding hierarchical information into text-based representations.

In contrast, our approach is fundamentally distinct, as it addresses hierarchical classification through the visual space. Rather than relying on hierarchical labels as prompts, we investigate how hierarchical classification can be linked to visual grounding, analyzing images at varying levels of detail—ranging from fine-grained to holistic representations. Additionally, we specifically linked detailed part-level segments to fine-grained labels and coarse segments to coarse labels, ensuring that each level of unsupervised visual segments contributes effectively to its corresponding learning process. This emphasis on visual grounding forms the core of our contribution, setting it apart from TransHP.

Additionally, our supervision strategy further differentiates the two approaches. While TransHP adopts a coarse-to-fine supervision framework, we reverse this by applying supervision from fine-grained to coarse labels from our consistent visual grounding motivation. This novel formulation enables a stronger and more consistent alignment between visual features and hierarchical levels, further underscoring the originality of our work.

In summary, while both methods incorporate coarse labels and share a ViT backbone, our focus on visual grounding and the fine-to-coarse supervision paradigm highlights a fundamentally different and innovative approach to hierarchical classification.

---

2. Experiments on larger dataset, iNat21-Mini

We want to make it clear that the lack of GPU resources was NOT used as an excuse to avoid conducting experiments. Rather, we simply requested additional time due to the constraints involved. The updated results for the iNaturalist dataset are now provided below.

It is also worth emphasizing that not all research environments have access to abundant computational resources, such as 8 A100 GPUs in the Cloud. Research conducted with limited GPU resources, while focusing on diverse experiments on smaller datasets, is no less meaningful and can provide valuable insights. We stand by the validity and contribution of our approach under these circumstances.

We'd like to share the results of our experiments on the large-scale dataset (iNaturalist 2021-mini). iNat21-mini [1] contains a total of 10,000 classes and 500,000 training samples and 100,000 test samples. For our experiments, we focused on a 3-level hierarchy consisting of order (273 classes), family (1,103 classes), and name (10,000 classes) for our 3-level hierarchy.

The results are presented in the table below. Compared to Hier-ViT, which uses the same ViT-small backbone, our method demonstrates the fine-grained accuracy improvement of over 7.29% and a 8.3% improvement in the FPA metric, representing a significant performance gain.

We hope these results address concerns about large-scale data. We will include this experimental result in the revised manuscript.

< iNat21-Mini (273 - 1,103 - 10,000) >

[1] Benchmarking representation learning for natural world image collections. 2021

11. Resource costs

11, How about the training and inference speed of the proposed method? ... it is necessary to provide a comparison of resource costs, including both time and memory usage.

As suggested by the reviewer, we have included training/inference time and memory usage to the table below. The time is measured per iteration with a batch size of 64 on an NVIDIA A40 GPU. In terms of memory usage, graph pooling reduces the size of the patches, leading to lower memory consumption compared to using patches of the same size without pooling. However, the additional computation required for superpixel generation and graph pooling increases the time compared to a standard ViT.

Our method achieves higher performance (>10%) than a standard ViT but comes with the trade-off of increased computation time. We have included this limitation in the Discussion section.

Additionally, the current implementations of graph pooling and superpixel generation are not fully optimized, and we expect future improvements with more efficient algorithms to address this limitation.

---

12. More thorough literature review

12. The literature review is far from complete. In addition to [a, b, c], numerous efforts in hierarchical scene parsing is totally missing; see related work section in [d, e].

We have revised the related work section to clearly distinguish our work from existing studies and to provide a more comprehensive overview of the field. Specifically, we have incorporated discussions on hierarchical semantic segmentation, including the works mentioned [d, e], as well as additional references to ensure coverage of hierarchical scene parsing efforts.

Due to space limitations, we have kept the main text concise while adding more detailed discussions in the Appendix. We believe this revision addresses the reviewer's concern and provides a clearer context for our contributions.

---

14. Minor issues

14. Minor issues include: a) the usage of the terms "interpretability" (L447) and "explainability." b) citation format. c) vector images. d) missing period after Eq. 2. e) the presentation of Eq. 3.

We have addressed all other points, but could you clarify what is meant by "a) the usage of the terms 'interpretability' and 'explainability' (L447)”? This clarification will help us ensure an appropriate response.

---

15. Conclusion Section

A Conclusion section should be added to properly conclude the work and offer insights of downside of impact of the work.

We have added a Conclusion section to summarize the work and discuss the limitations.

---

We hope this clarifies the reviewer’s concerns. If there are any further concerns or clarifications needed, we would be happy to discuss them further.

6. Comparison between Correct and Incorrect Predictions

6. Results in Fig. 5 do not totally make sense to me. Examples in a) and b) are not equally recognizable, ... One way for improvement is to examine for similar hard-level images ....

Initially, the images were selected randomly; however, in response to the reviewer’s suggestion, we have updated the PDF to include examples of images with comparable difficulty levels. These examples consistently demonstrate that correct predictions exhibit better clustering, while incorrect predictions often show fractured or misaligned groupings.

For example, in the updated PDF’s Figure 5, the first row shows two images where the model is tasked with recognizing a shoe. In the correct prediction case, the shoe and the sock are well-grouped (in green), showing coherent segmentation. In the incorrect prediction case, despite being a similar image, the model fails to recognize these parts, resulting in highly fractured segments.

While Figure 5 does not explicitly determine whether poor clustering is the cause or effect of incorrect predictions, it highlights a clear relationship that provides valuable insights into the model’s reasoning process. We have added additional examples in Appendix Figure 8.

---

7, 13. Justification of Tree-path KL divergence loss and comparison with other losses

7. .. It would be useful to compare the effectiveness of tree KL loss against these alternatives...

13. In fact, the hierarchical loss function in [d] is superior to the proposed Tree-PATH KL loss ..

To evaluate the effectiveness of our Tree-path KL Divergence loss, we compared it with two alternatives: Binary Cross Entropy (BCE) loss in [c] and Flat Consistency loss. BCE directly replaces the KL divergence component, while Flat Consistency loss, inspired by a bottom-up approach, infers coarse predictions from fine-grained ones and uses BCE to match them with the ground truth.

As shown in the table below, Tree-path KL Divergence loss outperforms both alternatives, achieving the highest FPA on the Living-17 dataset and demonstrating superior accuracy and semantic consistency. Similar trends are observed on the Aircraft dataset.

We have included these results and explanations in the updated manuscript Table 5 and 10.

<Living-17 dataset>

<Aircraft dataset>

Regarding the loss in [d], it appears to focus on pixel-level hierarchical segmentation tasks, which are not directly applicable to our instance-level classification setting. We hope this clarifies our approach and the scope of the comparisons.

---

10. Experiments on larger datasets

10. The datasets used for evaluation are small. larger datasets with complex hierarchy (ie.g., ImageNet-1K, iNaturalist) should be evaluated to better assess effectiveness.

We would like to clarify our dataset choices and experimental setup.

First, CUB, Stanford Cars, and Aircraft are among the most widely used benchmark datasets in prior hierarchical classification studies [1, 2, 3, 4]. We selected these datasets to ensure a fair comparison with existing methods. To further evaluate the effectiveness of our approach on a more diverse and challenging dataset, we conducted experiments on BREEDS, a subset of ImageNet, which includes a broader range of classes beyond a single type (e.g., birds or aircraft).

As for ImageNet-1K, its highly imbalanced hierarchy poses significant challenges for applying our approach. Consequently, most hierarchical classification studies on this dataset have focused on flat-level classification rather than multi-granularity approaches.

To further validate our method, we are currently conducting additional experiments on the iNaturalist dataset, which provides a larger and more complex test bed. We will update the results as soon as they become available. However, we kindly request your understanding, as our experiments are conducted on a single GPU system (Nvidia A40), which leads to longer training times for larger datasets.

[1] Your “Flamingo” is My “Bird”: Fine-Grained, or Not, 2021
[2] Consistency-aware feature learning for hierarchical fine-grained visual classification, 2023
[3] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024
[4] HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification, 2024

3. Larger hierarchies with Tree-path KL loss

It would be beneficial to address whether the proposed solution can effectively manage larger hierarchies, particularly those with depths of up to 10 or 20.

Thank you for the insightful question. Managing larger hierarchies with depths of 10 or 20 is indeed an important point to consider. While our current approach focuses on 2-3 levels, following prior hierarchical multi-granularity classification works [1, 2, 3], scaling to deeper hierarchies poses new challenges. Specifically, applying our current KL loss directly to such deep levels would likely encounter difficulties in maintaining effectiveness.

We acknowledge the need for scalability in deeper and more imbalanced trees and view this as a promising direction for future research. Exploring adjustments to the loss function and other architectural adaptations to handle larger hierarchies is an exciting area we plan to investigate further.

[1] Your “Flamingo” is My “Bird”: Fine-Grained, or Not, 2021
[2] Consistency-aware feature learning for hierarchical fine-grained visual classification, 2023
[3] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024

---

4. Visualization of Attention maps

... it is unclear why the model would “ensure that each hierarchical classifier focuses on the same corresponding regions” ... I expect visualizations of Grad-CAM results in different hierarchical levels from the proposed model.

We acknowledge that the model may sometimes use shortcuts by attending to different regions at different levels and still deliver correct predictions. Our method is designed to guide classifiers toward consistent visual grounding, but it does not directly enforce this behavior. To better reflect this, we will tone down our wording from “ensure” to “guide” in the manuscript.

Instead of Grad-CAM, we visualized attention maps from the transformer, as they provide a more direct representation of what the model attends to. The visualizations reveal that from lower to upper blocks, the model increasingly attends to similar regions. In the lower blocks, attention is more detailed and localized (e.g., snake’s head, parts of its body), while in the upper blocks, attention expands to include broader regions encompassing the areas highlighted by the lower blocks. These patterns align with our intended design for visual grounding in hierarchical classification.

We believe these visualizations validate our claim and have included them in the Appendix D.2.

---

5. 8. 9. Experimental Comparisons and Baseline Choices

5. The experiments are somewhat questionable... HRN, published in CVPR'22, is relatively outdated... several competitors [a-b] ...

9. More top-leading hierarchical classification work should be included in the comparison.

Thank you for introducing the new research [a, b]. We have added it to the related work section. While we aimed to compare our method with the most recent high-performing studies, regrettably, none of the latest works [a, b, c], including those you mentioned, had publicly available code. Also, this limitation is partly due to the focus of most studies on flat-level classification, resulting in few baselines for hierarchical multi-granularity classification. We hope our work inspires further research in this important area.

8. While FGN and HRN use ResNet-50 as the backbone, this work adopts ViT-S.

To account for differences due to backbone choices, we explicitly indicated the backbone architecture in the tables. As there were no existing hierarchical multi-granularity classification works using a ViT backbone, we introduced Hier-ViT to provide a ViT-based comparison. Additionally, as strong baselines, we trained flat-level classifiers and applied HiE [d] to improve fine-grained classification using coarse classifiers.

5. ... the experimental results of HRN differ from those reported in the original publication.

Upon review, we noticed that the results for the CUB and Aircraft datasets were reported using a batch size of 64 instead of 8. We have corrected this to the results with a batch size of 8. Thank you for your careful attention to detail. Also, slight differences occur because the original paper trained for 200 epochs, whereas we standardized all experiments to 100 epochs for fairness. Beyond this, we followed their codebase and experimental settings.

We hope this clarifies our experimental comparisons and addresses the reviewer’s concerns.

[a] HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification, 2024
[b] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024
[c] Consistency-aware feature learning for hierarchical fine-grained visual classification, 2023
[d] Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification, 2023

Dear reviewer khL4,
Thank you for your valuable feedback and comments. We appreciate your recognition of the sound motivation for visual consistency, the new metrics for hierarchical classification, and the state-of-the-art performance of our method. Your diverse and constructive comments have been instrumental in improving our work. We have also updated the PDF to reflect these improvements, and we kindly invite you to review the revised version. Below, we address your concerns and questions.

---

1-1. Quantitative Support for Consistent Visual Grounding

1. The key point of this paper, which is making classifiers at different levels attend to consistent visual cues, lacks support from quantitative results or theory.

To provide quantitative support for our observation in Figure 2, we analyzed Grad-CAM heatmaps of coarse and fine-grained classifiers. Specifically, we compute two metrics: the overlap score and the correlation score. The overlap score quantifies the degree to which the regions activated by the two classifiers coincide. The correlation score measures the linear relationship between the activation values of the overlapping regions in the two heatmaps. Higher overlap and correlation scores indicate stronger agreement between the regions attended to by the two classifiers. Conversely, lower scores highlight a lack of alignment in their focus.

In the Table below, results from the FGN model on the Entity-30 dataset show that, interestingly, when both classifiers made correct predictions, overlap and correlation scores were significantly higher. Conversely, incorrect predictions corresponded to notably lower scores. These findings support our motivation that aligning the focus of classifiers can enhance both accuracy and consistency. We have updated the detailed explanation and results in Appendix A.

1-2. Utility and Limitations of Grad-CAM and Transfer Rate

What is the transfer rate after adopting the proposed model? In addition, Grad-CAM is an approximation and does not truly explain how the network operates.

While Grad-CAM is an approximation, it is a widely used tool for observing class activation and provides valuable insights into model behavior. Our analysis demonstrates meaningful patterns that support the effectiveness of our proposed method. However, we acknowledge its limitations and will explore additional evaluation methods in future work.

If the reviewer refers to the improvement in consistency and accuracy after adopting our method, our experimental results already demonstrate superior and consistent performance across benchmark datasets, indirectly supporting the effectiveness of consistent visual grounding. If clarification is needed, we are happy to provide further details.

---

2. Novel Contribution over CAST

We thank the reviewer for the opportunity to clarify our contributions to hierarchical classification and the novel role of CAST in our work.

(1) Key Insight:
Our observation revealed that classification at different granularities involves fundamentally distinct tasks requiring attention to different but consistent regions within an image. We found that inconsistencies arise because each classifier tends to independently attend to different regions without connection. This observation led us to propose consistent visual grounding as a novel solution to connect hierarchical classifiers across levels.

(2) Leveraging Semantic Segments for Hierarchical Classification:
While we adopted CAST as part of our architecture, it is important to emphasize that CAST originates from a different task, weakly-supervised semantic segmentation. In CAST, “hierarchy” refers to “part-to-whole” visual grouping (e.g., eyes, nose, arms), while our work addresses a “taxonomy hierarchy” (e.g., bird - Green Hermit). It was NOT evident that the concept of “part-to-whole” segments would align well with a taxonomy hierarchy; this connection is a novel discovery introduced through our work.

In addition, based on our observation in (1), we newly propose leveraging segments at different granularities to enhance multi-granularity classification. To the best of our knowledge, the use of segments has NOT been applied to hierarchical classification tasks.

Thus, this adaptation is neither trivial nor an obvious solution; it stems from our novel observation and bridges two distinct fields to tackle challenges unique to hierarchical classification.

We hope this summary highlights the novelty and importance of our work.

Experiments on larger dataset, iNat21-Mini

Thank you for waiting. We'd like to share the results of our experiments on the large-scale dataset (iNaturalist 2021-mini). iNat21-mini [1] contains a total of 10,000 classes and 500,000 training samples and 100,000 test samples, structured within an 8-level hierarchy. For our experiments, we strategically focused on a 3-level hierarchy consisting of name, family, and genus. We made this choice because we believe that models with a meaningful level of granularity are more practical for real-world applications.

To elaborate, the number of classes at each level is as follows: Kingdom: 3, Supercategory: 11, Phylum: 13, Class: 51 Order: 273, Family: 1,103, Genus: 4,884, Name: 10,000

We deliberately excluded extremely coarse-grained levels like kingdom (3 classes), as such distinctions offer minimal practical value for classification tasks. Likewise, overly fine-grained levels such as genus (4,884 classes), where many species are represented by only one or two samples, fail to offer meaningful differentiation from direct name-level classification. Thus, we selected order (273 classes), family (1,103 classes), and name (10,000 classes) for our 3-level hierarchy. This choice ensures that each higher-level class meaningfully represents a diverse yet relevant subset of lower-level classes, enabling both meaningful classification and the evaluation of consistent predictions.

The results are presented in the table below. Compared to Hier-ViT, which uses the same ViT-small backbone, our method demonstrates the fine-grained accuracy improvement of over 7.29% and a 8.3% improvement in the FPA metric, representing a significant performance gain.

We hope these results address concerns about large-scale data. Also, while our experiments focused on a meaningful 3-level hierarchy, as previously addressed in our response to Comment 3, we believe that designing a model capable of efficiently scaling to deeper hierarchies represents an important and promising direction for future work. We will include this experimental result in the revised manuscript.

< iNat21-Mini (273 - 1,103 - 10,000) >

[1] Benchmarking representation learning for natural world image collections. 2021

We appreciate the reviewer’s feedback and would like to clarify and address the following concerns.

<Experiments>

While some recent works are not included due to the lack of publicly available code or model checkpoints, it is important to note that all these works are also based on ResNet backbones. Given this, we believed it was more appropriate to include a ViT-based baseline rather than another ResNet-based one. Furthermore, since no prior works in multi-granularity classification used a ViT backbone, we developed Hier-ViT to fill this gap.

Regarding training epochs, we standardized all experiments to 100 epochs to ensure fair comparisons. FGN originally used 100 epochs, so we maintained this setting, and we reduced HRN from 200 epochs to 100 for consistency. Standardizing training epochs is a widely accepted practice for fair evaluation. Additionally, our results for FGN show improved performance compared to the results reported in their paper, and for HRN, the performance differences are minimal, ranging from 0.09 to 0.47.

For HRN, we included experiments with a larger batch size, along with the original batch size of 8, because the method exhibited significant sensitivity to batch size, a notable observation that highlights its behavior compared to other methods.

Based on these considerations, we respectfully disagree with the claim that our experimental setup is unconvincing. We believe our decisions ensure a fair and meaningful comparison.

---

<Loss in Hierarchical Segmentation [d]>

We appreciate the opportunity to clarify our statement regarding the loss in [d]. While it is indeed possible to adapt the loss in [d] for instance-level classification by treating instances as analogous to pixels, our primary intention was to emphasize the fundamental differences in focus between the two tasks.

In [d], during inference, each pixel $i$ is associated with the top-scoring root-to-leaf path in the class hierarchy $T$. These root-to-leaf paths are predefined, and the task focuses on selecting the best path. As a result, the loss in [d] is designed to emphasize the weakest predictions along the path to improve overall accuracy. In contrast, our objective is to predict each node along the root-to-leaf path individually, ensuring that all predictions are both accurate and consistent across the hierarchy. The key challenge in our task lies in addressing potential inconsistencies between hierarchy levels, and we propose a method specifically designed to resolve this issue.

Specifically, the loss in [d] prioritizes the most violated hierarchical constraint for each score vector through the "min" operation in Equation (6). That means, if the loss (6) in [d] is applied, the model may prioritize resolving constraints for the fine-grained taxonomy (e.g., "Green Hermit") at the expense of optimizing the coarse taxonomy (e.g., "bird"). In contrast, our approach is designed to simultaneously predict multiple taxonomies across the hierarchy (e.g., "bird" and "Green Hermit"). Thus, in our approach, we model all classes at each level as distributions and adopt a KL divergence loss to encourage balanced learning across all taxonomy levels. This ensures a holistic approach that aligns with the multi-granularity objectives of our task.

Nonetheless, we acknowledge the potential utility of explicitly enforcing hierarchical constraints through the loss in [d] and will conduct experiments to evaluate its applicability in the instance-level setting.

---

<Concept hierarchy>

We appreciate the reviewer’s comment and would like to clarify the distinction between part-to-whole hierarchies and taxonomy hierarchies, as well as how our approach bridges these concepts. While both are conceptual hierarchies, their foundations and applications are fundamentally different.

A part-to-whole hierarchy represents a spatial compositional hierarchy, where smaller parts (e.g., eyes, nose, arms) combine to form a larger whole (e.g., a face or body). This type of hierarchy is grounded in visual composition and spatial relationships. In contrast, a taxonomy hierarchy (e.g., "bird" → "Green Hermit") is a semantic hierarchy, structured by coarse-to-fine class relationships that are defined by meaning and semantics, not by spatial composition.

Our work bridges these two ideas by incorporating visual grounding concepts—which are typically applied in spatial compositional hierarchies (segmentation tasks)—to address challenges in semantic taxonomy hierarchies (hierarchical classification task).

On that distinction, exiting works only enforce the consistency along the semantic hierarchy, whereas ours is the only one that ground the consistency of semantic hierarchy on the visual spatial parsing consistency. As a consequence, our work outperforms Hier-ViT (a ViT-based model that enforces only semantic consistency) by more than 10%, a significant margin, and SOTA (HRN) by over 4.25-6.36% on BREEDS, subset of ImageNet with more diverse categories.

On the benchmark of hierarchical classification, we deliver the significant gain with the first (unsupervised) visually grounded classification model. Our experimental validation is solid and our model stands out in novelty as a single such paper on the topic of hierarchical classification.

We urge the reviewers not to let this seeming conceptual resemblance overlook our significant contributions on both accounts (practical results and vision insight).

Dear reviewer YARY,
Thank you for your valuable feedback and detailed comments. We greatly appreciate your thorough understanding of our work and your thoughtful recognition of its strengths. Your remarks on the clarity of our problem setup, method design, and experimental results are deeply encouraging. We address your concerns and questions in the response below.

---

1. Interpretation of Part 4.4

In part 4.4, I'm not sure how to interpret the result. .... But to me, this just shows that superpixel segmentation is not necessarily adapted to a semantic problem, because it's done at the color level, and semantic in images is not necessarily associated with color.

As you mentioned, the purpose of the superpixel segmentation is not to directly associate colored segments with specific semantics. Instead, we use it to evaluate whether the model effectively groups the desired object, which indirectly reflects the quality of its predictions.

We replaced the example images with ones that are easier to understand. For example, in the updated PDF’s Figure 5, the first row shows two images where the model is tasked with recognizing a shoe. In the correct prediction case, the shoe and the sock are well-grouped (in green), showing coherent segmentation. In the incorrect prediction case, despite being a similar image, the model fails to recognize these parts, resulting in highly fractured segments.

While superpixel segmentation operates at the color level and may not fully capture semantic meaning, these examples help illustrate how the model’s focus aligns with its predictions. This provides indirect interpretability, allowing us to understand the reasoning behind correct and incorrect predictions.

We hope this explanation clarifies the intent and interpretation of these results. Please let us know if further clarification is needed.

---

2. Modification of Figure 1 and 4

In Figures 1 and 4, I find that the space between class names (like "Chimpanzeeferret" in Figure 1) and their position/alignment in the tree can be difficult to read. ...

Thank you for the careful review. We have revised it accordingly.

---

3. Reflection on Fine-to-coarse design

Thank you for the insightful reflections. Your points on the coarse-to-fine direction are valid, and we appreciate the opportunity to further clarify the rationale behind our fine-to-coarse approach.

Firstly, as the reviewer mentioned, if the hierarchical structure of labels is a tree, it could make sense to classify the fine level first and infer the higher levels by traversing up the tree (i.e., a flat classification approach). However, as you correctly noted, a single error at the fine level would invalidate all higher-level predictions. This highlights the need for a model capable of classifying all labels within the hierarchy.

As we included the experiments on Coarse-to-Fine and Fine-to-Coarse architectures in Table 3, we carefully considered how to design the architecture effectively. Since taxonomies often follow a tree structure, with broader classes leading to finer categories like genus and species, the coarse-to-fine approach naturally aligns with human perception and has been widely adopted in prior works.

However, taxonomy is a human construct, and it made us question whether machine learning models should necessarily process information in a coarse-to-fine manner. When we reflect on how we learn abstract/coarser concepts, we often find that abstract categories can be learned more easily by learning specific concepts. For instance, seeing Siberian Huskies, Chihuahuas, Malteses and so on could lead to the broader category "dog" based on shared features like being "cute, four-legged animals."

This perspective, along with the structural design we adopt from CAST, inspired us to explore a Fine-to-Coarse learning strategy, where fine features are aggregated to form higher-level features (segments). Surprisingly, this approach achieved strong performance in our experiments, suggesting its potential for hierarchical classification.

We believe the best model for hierarchical classification is still underexplored, and as you suggested, there is much potential to investigate more direct and diverse approaches. Thank you again for the thoughtful discussion—it has inspired us to think further on these possibilities.

6. Claim in 4.6

The claims in 4.6 are similar to the insights provided in [1].

What we highlighted in Section 4.6 is the finding that taxonomy class supervision can be beneficial for segmentation. In CAST, the hierarchy refers to part-to-whole segments, and it was unexpected that taxonomy hierarchy could improve this. If our explanation is unclear, we would appreciate it if you could elaborate on what you mean by "similar insights" in [1], so we can address it more effectively.

---

Limited contribution

Overall, the technical contribution, evaluation, and insights provided by this work are all limited.

We have addressed our novel contribution in 1. To further clarify and strengthen our contributions, we have made several updates to the paper:

1. Related Work Revision: We revised the related work section to clearly position our work within the context of multi-granularity classification and to distinguish our direction from related research areas.

2. Support for Motivation: To reinforce our motivation, we conducted a Grad-CAM analysis, demonstrating the relationship between consistent visual grounding and improved hierarchical classification.

3. Additional Experiments: We added attention map visualizations to provide further interpretability of the model’s predictions. Additionally, we included a loss ablation study to evaluate the effectiveness of our Tree-path KL Divergence loss compared to other commonly used loss functions.

We believe these updates provide a more comprehensive understanding of our technical contributions, evaluation, and insights.

---

We hope this clarifies the reviewer’s concerns. If there are any further concerns or clarifications needed, we would be happy to discuss them further.

4. Cross-entropy over KL divergence loss

The motivation for using KL divergence instead of cross-entropy is unclear.... Both experiments and the theoretical explanation should be provided.

The motivation for using KL divergence instead of cross-entropy lies in the need to encode semantic consistency across hierarchical levels. KL divergence enables us to model the entire hierarchical tree as a single distribution, capturing the relationships across all levels simultaneously, rather than treating each level as an independent classification task, as is common with cross-entropy loss. To evaluate the effectiveness of our Tree-path KL Divergence loss, we conducted experiments comparing it to two alternatives:

• Binary Cross Entropy (BCE) loss: A widely used approach for hierarchical classification when treated as a multi-label classification task.
• Flat Consistency loss: Inspired by bottom-up approaches, it infers coarse-level predictions from fine-grained ones and applies BCE to align them with the ground truth.

As shown in the table below, Tree-path KL Divergence loss outperforms both alternatives, achieving the highest FPA on the Living-17 dataset and demonstrating superior accuracy and semantic consistency. Similar trends are observed on the Aircraft dataset.

<Living-17 dataset>

<Aircraft dataset>

We have included these results and explanations in the updated manuscript Table 5 and 10.

---

5-1 Comparison to Hierarchical Counterparts and more recent works

i) The comparison to hierarchical counterparts only contains out-of-date methods published before 2022 and the baseline (focusing on segmentation rather than classification). The top-leading solutions (e.g., [5]) are all ignored.

Our evaluation includes comparisons with relevant hierarchical classification methods such as FGN and HRN, which are widely recognized benchmarks in hierarchical multi-granularity classification. Additionally, we incorporated HIE (NeurIPS 2023), a more recent method, to provide an updated comparison.

Among the various directions in hierarchical classification, our work focuses on multi-granularity classification, where predictions are made simultaneously across multiple levels. In contrast, many existing methods (e.g., [5]) focus on flat classification, using coarse labels to enhance fine-grained predictions. Consequently, there are few methods available for direct comparison in multi-granularity classification.

In addition, recent works on multi-granularity classification [a, b, c] have not made their code publicly available, which is why we compared against FGN and HRN. Although the goal of TransHP [5] differs from ours, we attempted to evaluate it for comparison. However, [5] requires significant resources, such as training on 8 A100 GPUs, while our server is limited to a single A40 GPU, making it challenging to reproduce their results during this rebuttal period. We kindly ask for your understanding regarding this limitation.

5-2. focusing on segmentation?: We included CAST in our comparisons because we adopted it for visual grounding to address inconsistent predictions in hierarchical classification. We used it as one of the flat-level baseline. Additionally, since our work utilizes unsupervised segments to enhance hierarchical classification, it naturally raises the research question of whether this approach could also benefit segmentation in reverse. To explore this, we conducted additional experiments to evaluate its potential impact on segmentation tasks.

5-3. Shallow hierarchy

We acknowledge that the label hierarchy in our work is relatively shallow, with up to 3 hierarchical levels. However, this follows the standard practice in multi-granularity classification tasks, as seen in prior works [a, b, c].
We acknowledge the need for scalability in deeper trees and view this as a promising direction for future research. Exploring adjustments to the loss function and other architectural adaptations to handle larger hierarchies is an exciting area we plan to investigate further.

[a] Consistency-aware feature learning for hierarchical fine-grained visual classification, 2023
[b] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024
[c] HLS-FGVC: Hierarchical Label Semantics Enhanced Fine-Grained Visual Classification, 2024

Dear reviewer ZNGF , Thank you for your valuable feedback and comments. We appreciate your recognition of the clarity of our presentation and the significant improvements over the baseline. We address your concerns and questions in the response below.

1. Novel Contribution over CAST

The technical contribution is not enough. Specifically, the proposed method contains visual consistency and semantic consistency. However, the major design and implementation of visual consistency are directly borrowed from CAST [1]....

We thank the reviewer for the opportunity to clarify our contributions to hierarchical classification and the novel role of CAST in our work.

(1) Key Insight:
Our observation revealed that classification at different granularities involves fundamentally distinct tasks requiring attention to different but consistent regions within an image. We found that inconsistencies arise because each classifier tends to independently attend to different regions without connection. This observation led us to propose consistent visual grounding as a novel solution to connect hierarchical classifiers across levels.

(2) Leveraging Semantic Segments for Hierarchical Classification:
While we adopted CAST as part of our architecture, it is important to emphasize that CAST originates from a different task, weakly-supervised semantic segmentation. In CAST, “hierarchy” refers to “part-to-whole” visual grouping (e.g., eyes, nose, arms), while our work addresses a “taxonomy hierarchy” (e.g., bird - Green Hermit). It was NOT evident that the concept of “part-to-whole” segments would align well with a taxonomy hierarchy; this connection is a novel discovery introduced through our work.

In addition, based on our observation in (1), we newly propose leveraging segments at different granularities to enhance multi-granularity classification. To the best of our knowledge, the use of segments has NOT been applied to hierarchical classification tasks.

Thus, this adaptation is neither trivial nor an obvious solution; it stems from our novel observation and bridges two distinct fields to tackle challenges unique to hierarchical classification.

We hope this summary highlights the novelty and importance of our work.

---

2. Related work on hierarchical segmentation

The review only contains hierarchical image classification. However, pixel-level hierarchical classification (i.e., hierarchical image segmentation [2,3,4]) can also provide insights for this work. ...

Thank you for introducing the related work. We have revised the related work section and included discussions on hierarchical semantic segmentation to provide a more comprehensive overview.

---

3. Concatenating labels in Tree-path KL divergence loss

Concatenate labels from all levels to create a distribution has already been explored in [3,4]. The difference is [3,4] using cross-entropy loss, while this work uses KL divergence.

While [3,4] concatenate labels from all levels, our method differs fundamentally in both motivation and application. To clarify, hierarchical segmentation and hierarchical classification address entirely different challenges:

Hierarchical segmentation focuses on spatial granularity, identifying visual elements at different scales within the image (e.g., parts like "eye" or "head" versus the whole "person").

Hierarchical classification, on the other hand, deals with semantic granularity, where the image remains the same, but the interpretation of its content varies by level (e.g., "bird" → "hummingbird" → "green hermit"). A key challenge in hierarchical classification is addressing inconsistencies in predictions across levels (e.g., the coarse-level classifier predicts "plant," while the fine-level classifier predicts "bird").

For example, in HIPIE [3], instance class names (e.g., "person," "cat") are concatenated with part class names (e.g., "head," "eye") to capture a compositional hierarchy in spatial segmentation. In contrast, our method models semantic relationships as a distribution by one-hot encoding hierarchical labels and concatenating them.

Our primary goal is to ensure semantic consistency across levels in hierarchical classification. To achieve this, we introduce the Tree-path KL Divergence loss, which transforms hierarchical labels into a distribution and enforces alignment across the taxonomy. This motivation fundamentally differs from [3,4], by explicitly modeling and preserving the semantic relationships between hierarchical levels.

---

Dear reviewer d3Ey,

Thank you for your valuable feedback and comments. We appreciate your recognition of the motivation behind the hierarchical focus, the strong results compared to prior works, and the insights provided by the ablation studies. We address your concerns and questions in the response below.

---

To address questions (1) and (2), we would like to clarify the two main directions in hierarchical classification:

1. Flat Classification: This approach assumes a known taxonomy and focuses on fine-grained (flat-level) classification. Coarse labels are used during training to improve fine-grained predictions. At inference, higher-level taxonomy is derived in a bottom-up manner from fine-grained predictions. The output is a single label, and most existing works adopt this approach. While effective for detailed and clear images (e.g., close-up shots of birds), it can struggle with less distinguishable objects, as errors at the fine-grained level often lead to incorrect predictions at higher levels.

2. Global (Multi-granularity) Classification: This approach, which includes our work, predicts the entire taxonomy, addressing the limitations of fine-grained classification. By providing higher-level classifications, it offers more flexibility and robustness in real-world scenarios with ambiguous or partially visible objects.

---

(1) Supervision Assumption

(1) In L61, the difference in available labelling is presented as one of the motivations for hierarchical classification. However, the presented method assumes that all levels of the hierarchy are available. Can the technique work if the finest levels of supervision are not available?

In L61-62, we aimed to emphasize the importance of full-taxonomy classification under realistic scenarios. Specifically, L61 illustrates cases where coarse labels, like "bird," may suffice for some users, but experts require finer distinctions. We have revised this section to make it clearer (L33-L39). For this work, we assumed all labels are available during training (fully supervised). Your inquiry about the absence of fine-level labels during training is valid and highlights an important area for future exploration. Semi-supervised scenarios, where some labels are unavailable, represent another interesting and challenging problem that could extend this work.

---

(2) Flat Baseline Comparison

(2) Similarly, given the availability of the finest-level label, the other course levels in the tree are implied, so perhaps a more appropriate flat baseline would be a ViT that only predicts finest-level classes (and thus parent nodes by simple aggregation). It would also provide a more appropriate comparison in terms of architecture.

Your observation about the baseline aligns with the bottom-up inference in fine-grained classification. We have already included flat-level baselines trained at each level, as shown in Tables 2, 11, 12 in updated pdf (Flat-ViT/Flat-CAST). For this discussion, we can focus on the fine-level results, assuming that fine-level predictions are aggregated to derive the parent-level predictions. For instance, in Table 2, in the Flat-ViT case for Living-17, the fine-level accuracy of 72.06% propagates to the coarse level, resulting in 100% consistency. However, this consistency comes at the cost of overall accuracy (e.g., FPA: 72.06, Coarse: 72.06, wAP: 72.06, TICE: 0).

---

(3) Variance for some key results

(3) Given the relatively "small" sizes of the datasets by modern standards and some occasional closeness to the flat baselines in the scores (Tab 2.) Would it be possible to include some measure of variance for some key results?

To address the reviewer’s concern about variability, we trained and Hier-ViT, and H-CAST five times on the 2-level hierarchy Living-17 dataset and the 3-level hierarchy CUB dataset and. For each run, we randomly selected 90% of the original training data and used different random seeds to ensure variability. The slight drop in performance observed is attributable to the reduced training data (90% of the full dataset). The results, presented in the table below, show that H-CAST consistently achieves strong performance with low variance, which aligns with our previously reported findings.

(1) The intuition.

The claim that our intuition is the same as TransHP is incorrect. TransHP encourages semantic consistency through hierarchical prompting but does not enforce visual consistency. Additionally, Fig. 5 in TransHP only visualizes correctly predicted cases, but this is not unique to TransHP—prior works, including [6] (See Fig. 5), also show correct cases.

In contrast, we specifically investigate when inconsistencies occur and how to resolve them. Our analysis reveals that coarse and fine classifiers often attend to entirely different regions for the same image, leading to misaligned predictions. To address this, we explicitly encourages visual consistency, ensuring that classifiers remain visually aligned across hierarchy levels. Rather than just enforcing semantic consistency, we leverage visual segments to correct inconsistencies, making our approach fundamentally different.

Thus, our intuition is distinct from TransHP, as we focus on visual grounding rather than semantic refinement.

[6] Where to Focus: Investigating Hierarchical Attention Relationship for Fine-Grained Visual Classification, ECCV, 2022.

---

(2) The realization.

(1) Line 258-265 and Equation (1): Using cross-entropy loss for hierarchical supervision is a widely adopted practice [Equation (3) in [6], Equation (1) in [7]]. However, the uniqueness of each approach lies in its architectural design, block structure, feature utilization, and additional loss formulation. Claiming that two methods are identical solely based on shared equations oversimplifies the distinctions and fails to acknowledge these critical differences.

(2) Regarding TransHP, we have already discussed its relevance in the Related Work section, where we believe it is most appropriate. In the camera-ready version, we have further expanded the discussion on TransHP in the Experimental section to provide additional clarity.

(3) Line 266 (Methodology section): The discussion focuses on the difference between our method (H-CAST) and CAST because H-CAST adopts CAST's architecture to leverage visual segments. Since our methodology is built upon this design choice, CAST is the most relevant comparison in this section.

(4) Lines 267–269: Your concern is unclear. This section explains our design choice—Fine-to-Coarse supervision, which contrasts with prior methods ([6], [7], and even TransHP) that follow the Coarse-to-Fine direction. The intent is to highlight this methodological difference, not to claim ownership of a general concept.

[6] Where to Focus: Investigating Hierarchical Attention Relationship for Fine-Grained Visual Classification, ECCV, 2022.
[7] B-CNN: Branch Convolutional Neural Network for Hierarchical Classification, 2017.

---

(3) Main figure

As previously explained during the rebuttal, Figure 4 (2) in TransHP aligns more closely with Hier-ViT, another baseline we consider, rather than H-CAST. Unlike H-CAST, it lacks visual segments and TK loss—key components of our method.

The difference is evident in performance: H-CAST improves accuracy by +2.92pp on iNat-18, nearly four times TransHP’s +0.75pp gain over the previous SOTA. This demonstrates the effectiveness of visual grounding with segments and TK loss.

Thus, equating H-CAST with TransHP’s Figure 4 (2) is inaccurate, as our methodological differences lead to significantly stronger performance.

Addressing Unprofessional and Baseless Reviewer Critiques

Despite our repeated explanations and provided experiments, the reviewer continues to misrepresent our work with baseless accusations and unprofessional language, such as "content laundering" and "What makes me angry is..." A scientific review should be based on evidence and objective critique, not emotional reactions or unfounded allegations of misconduct.

Also, accusing us of making "the excuse of no GPUs" is unjustified, as we clearly explained the time constraints and later provided all results. We are not responsible for the reviewer's frustration caused by a refusal to engage with our clarifications in good faith. We expect reasoned, evidence-based discussion, not misinterpretations and inflammatory claims.

---

---

Before addressing the reviewer's comments, we reiterate the two key differences, as already explained in the rebuttal.

(1) Problem Scope:

TransHP focuses on fine-grained prediction using coarse labels (input: taxonomy, output: single-level fine-grained prediction), while H-CAST addresses multi-granularity classification, predicting the entire taxonomy (input: taxonomy, output: whole taxonomy). The key challenge is ensuring consistency across levels (e.g., avoiding mismatches like "plant" as coarse and "hummingbird" as fine).

We follow prior research [1,2,3,4], using standard benchmarks (CUB, Aircraft, BREEDS) and evaluating accuracy across all levels and consistency metrics (TICE, FPA). In contrast, TransHP focuses only on fine-grained accuracy, leveraging coarse labels as intermediate outputs and comparing against HiMulConE [5], which uses contrastive learning with additional coarse labels.

The differing objectives lead to distinct evaluation metrics, benchmarks, and baselines. While both methods use hierarchical taxonomy, they belong to separate research domains, as clarified in the updated related work section.

---

(2) Methodology:

TransHP operates in the semantic space, using coarse labels as prompts to refine fine-grained classification. H-CAST, however, emphasizes visual parsing consistency, aligning visual representations with hierarchical structures across levels—from fine-grained parts to holistic scenes.

H-CAST links part-level segments to fine-grained labels and coarse segments to coarse labels, ensuring unsupervised visual segmentation contributes meaningfully at each level. This shift from semantic-space consistency (TransHP) to visual parsing consistency (H-CAST) represents a fundamental methodological distinction in hierarchical classification.

[1] Your "Flamingo" is My "Bird": Fine-Grained, or Not, 2021
[2] Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification, 2022
[3] Consistency-aware Feature Learning for Hierarchical Fine-grained Visual Classification, 2023
[4] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024
[5] Use All the Labels: A Hierarchical Multi-Label Contrastive Learning Framework, 2022

---

(3) Additional results of TransHP

Additionally, although the problem scope differs, we included TransHP as a ViT-based baseline in the camera-ready version because it generates coarse labels as intermediate outputs, as requested by the reviewer.

The results show that H-CAST significantly outperforms TransHP in both accuracy and consistency (FPA: 74.35% → 85.12%, Top-1 Fine-grained Accuracy: 76.65% → 85.24%). Meanwhile, Hier-ViT, a TransHP variant (Fig. 4 (2)), performs slightly worse than TransHP, supporting their claim.

Furthermore, on iNaturalist-2018, the exact dataset used in TransHP, H-CAST achieves strong top-1 accuracy, confirming its effectiveness. (Here, H-CAST was trained as a small model for 100 epochs.)

Thus, H-CAST is NOT a variant of TransHP, and the consistent visual grounding and TK loss we introduce are highly effective.

For optimal setup, we used the official codebase, training for 300 epochs (H-CAST: 100 epochs). Prompt block placement for coarse-level supervision followed Table 1 in the TransHP paper:

• 2-level datasets: Selected the better-performing configuration between [6, 11] and [8, 11].
• 3-level datasets: Used [6, 8, 11] blocks.

(4) Experiments

→ (1) We did not avoid specific datasets due to GPU limitations. During the review period, we mentioned that large-scale dataset experiments would take longer due to GPU constraints and requested patience so that we could first proceed with the discussion. A few days later, we updated the results. Below is the exact statement we provided at that time:

"We are currently conducting additional experiments on the large-scale iNaturalist dataset to further validate our method on a larger dataset. We will update the results as they become available. However, we kindly ask for your understanding, as our training environment relies on a single GPU (Nvidia A40), which causes the experiments to take longer to complete."

(2) As discussed earlier, our task is fundamentally different from single-level fine-grained classification. We focus on multi-granularity classification, which requires different benchmark datasets. Thus, we followed prior work in this research line and adopted the standard datasets used in this area.

(3) Additionally, FGN and HRN remain strong baselines in this field. They are not simply "old methods"—both have demonstrated competitive performance and have publicly available codebases. As shown in our results, HRN even outperforms TransHP. Additionally, more recent works in multi-granularity classification ([3], [4]) have not released their code, making FGN and HRN the most practical and reproducible baselines for comparison.

[3] Consistency-aware Feature Learning for Hierarchical Fine-grained Visual Classification, 2023
[4] Hierarchical multi-granularity classification based on bidirectional knowledge transfer, 2024