V-CECE: Visual Counterfactual Explanations via Conceptual Edits

Thank you for taking the time to review our submission. Below, we respond to the main concerns raised under the “Weaknesses” and “Questions” sections.

1. We would like to respectfully disagree with the claim that the human study “has enormous flaws,” or that important information regarding the human survey is missing, even from the appendix (“completely unacceptable”). We believe these characterizations do not accurately reflect the content of the submission. The points raised, including the number of participants, the annotation process, the materials shown, and the methodology used, are in fact clearly presented in the paper in Appendix E. The section provides detailed information about the setup: it involved 31 annotators; the platform used for the survey (we develop a custom platform using Label Studio); it includes full instructions and example screenshots of the annotation interface; and it describes the exact questions asked. The stimuli and evaluation protocol are also explained. In our view, this information sufficiently documents the human study and is in line with the level of detail expected for an evaluation of this kind. Regarding the absence of a null hypothesis or power analysis; our study is exploratory in nature, aiming to assess semantic interpretability and not designed as a confirmatory statistical test. This is consistent with the way similar human studies have been conducted in [1, 2] that we are directly comparing our work with, where such statistical elements, including null hypotheses and power analyses, were likewise not included. The structure, scope, and reporting of our human evaluation are directly aligned with the format and methodology used in [1, 2], which have been peer-reviewed and accepted at top venues (IJCAI, ICML). We did not design a new or ad hoc procedure; instead, we deliberately followed established, community-accepted protocols. Thus, we do not believe the study contains methodological flaws, and we consider the information reported to be appropriate and complete for the goals it serves.

2. V-CECE introduces a general method to explain any pre-trained classifier in semantic terms, thereby revealing the semantic gap between human and model interpretation; it is not limited to a specific recipient or domain. However, the classifiers we study, CNNs, ViTs, and LVLMs, are among the most widely used architectures in both research and practical applications. In particular, CNNs and ViTs have served as the backbone of image classification for years, making black-box semantic explanation methods for them both timely and important. Similarly, LVLMs are increasingly deployed across domains due to their strong performance and generality, yet they remain opaque by design, warranting targeted explanation endeavors. To this end, V-CECE is designed to be model-agnostic: practitioners can plug in any model to obtain semantic explanations and generate counterfactual images. Beyond previous work, V-CECE provides a novel insight: we uncover and analyze the explanatory gap in generated counterfactuals, highlighting a key mismatch between model behavior and human-understandable semantics. We also include a real-world use case, autonomous driving, where semantic transparency is critical. For example, in a system tasked with deciding whether a vehicle should stop or go, semantic failure could lead to unsafe behavior. Our results show that some classifiers (e.g., DenseNet) may fail in this setting due to limited semantic perception. This finding underscores the importance of deploying semantically-aware models in safety-critical applications and may inform future practices in the design of explainable autonomous systems.

Minor Points

1. We will change this in the camera ready, should the paper be accepted.

2. We cite [5] as a theoretical claim of the importance of semantics in explainability. However, this claim has been proven in practice: the works of [1, 2] with which we compare with, are based on semantic counterfactuals, supporting the claim that semantics are crucial for explainability. These works have been published in top-tier conferences (IJCAI, ICML) verifying the validity of this claim. It is also a highly influential paper in the domain of explainability in other works, published in CVPR and ECCV as well [3], [4]. The human study (machine teaching) presented in [2] reveals that semantics are not only important but often adequate to explain differences between classes according to humans. Therefore, there is some extra evidence regarding the importance of semantics in explainability apart from [5], which is once again verified in our work, since we reveal that given a semantic edits algorithm, some classifiers disagree with humans regarding the label-flipping step.

3. This is not what this sentence attempts to point-out; we do not claim that prior works do not have sufficient code bases. Instead, we mean that they are not reproducible by humans in the sense that a human cannot easily detect what has changed in comparison to the source image when generated edits are dispersed or targeted to non-semantic areas. At the same time, we mention that counterfactual generations are not interpretable; a human cannot accurately reproduce what cannot be interpreted from their point of view.

4. We will use bookends in the camera-ready.

Questions

1. All requested details are already included in Appendix E. However, if additional clarification is needed, we are happy to provide it.

2. Our human study targets to reveal the explanatory gap between classifiers and humans, given different pre-trained classifiers. It does not target to compare generated images between different methods. To the best of our knowledge, no prior work in counterfactual generation makes such a claim. Therefore, there would be no purpose in including generated samples from prior works, since their goal is different. Other than that, this would be unfair for some prior works, since they mostly target label-flipping, but their edits may not be localized, therefore it is harder for humans to assess when label-flipping occurs and why. In this case, the human study would be questionable, as it targets human perception regarding label-flipping; if humans are unable to accurately point the step where the label changes, the results are unreliable, therefore the findings would obscure the semantic level of human understanding. Even if we wanted to directly compare our method to others using a human survey, this would not be possible. The algorithms proposed by [1, 2] do not generate images, but instead provide only theoretical edit suggestions. Similarly, TIME does not support iterative editing; instead, it modifies all relevant areas at once. As a result, it is not possible to isolate and evaluate each individual edit step in a way that would be meaningful for human annotators. Therefore, a direct human-based comparison would be infeasible and potentially misleading.

3. We presented a practical use-case regarding autonomous driving. Revealing that some popular models, such as DenseNet, diverge from human semantic understanding in this application is crucial and calls for the deployment of semantic-aware models. For example, when engineers would like to deploy a new classifier for a self-driving system, they could check its semantic awareness using VCECE as part of a reliability check pipeline, and compare it to well-known classifiers used in our study (for example, is the new classifier closer to Claude 3.5-the best performer- or closer to CNNs-worst performers- regarding the number of edits?). The reported degree of semantic awareness would suggest improvements in model architecture, if it is found to be non-satisfactory.

We hope these clarifications have addressed the reviewer’s comments and helped resolve any concerns regarding the paper. Thank you once again for your time and effort in reviewing our submission.

[1] Dervakos, Edmund et al. Choose your data wisely: A framework for semantic counterfactuals. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23), pages 382–390..

[2] Dimitriou, Angeliki et al. Structure your data: Towards semantic graph counterfactuals. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), volume 235.

[3] Zemni, Mehdi, et al. "Octet: Object-aware counterfactual explanations." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.

[4] Jacob, Paul, et al. "STEEX: steering counterfactual explanations with semantics." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

We would like to respectfully clarify that the main paper does include a direct reference to the relevant Appendix section in the Ηuman Evaluation paragraph (Line 250).

Ah I missed this as appendix is abbreviated

If the reviewer could help clarify certain points of grammatical errors, we would appreciate it in order to make the submission more complete for the CR version.

Really just minor things, for example the appendix uses incorrect quotations on line 466, alongside my earlier comments. A careful proofread will be enough for this.

As for the use of raster images, this is a formatting issue that can easily be resolved during the CRC phase. It was not due to lack of time or oversight but simply a technical choice that will be corrected. We will rectify this, as mentioned in our previous response, for the CR version.

Thanks

Regarding the human study, we would like to emphasize that “ML applications” is a broad field, and participant counts in human studies can vary depending on the specific task. In our case, a more appropriate comparison is with human evaluations conducted in recent works focused on counterfactual generation, where participant numbers are often in the same range as ours. Specifically, recent papers published at top-tier venues such as ICML, IJCAI and CVPR report similar participant counts (~30) [1, 2, 3], when such human studies are included at all [4]. Thus, we do not believe that our sample size is on the lower end for this area, but rather in line with current standards in similar works, encouraging fair comparison.

I think that pointing to other cherry picked studies is a weak argument here, I can also point to other studies which do more rigorous statistical evaluation in the same conferences. My point is not the sample size specifically, but the combination of “low” sample with no power analysis. If you had a sample size of 100+ then effect size will take over (i.e., all p values become significant) and you can just use this as your metric. But for lower samples, you really need to do a proper power analysis and statistical test. Please keep this in mind for future work, I want the field to progress, user testing is important, but we have to do it correctly. Your results seems significant reported as means, but that is not enough for me to say 100%.

As for the appendix, the screenshot serves as an example to help readers understand the structure of the survey interface and the type of stimuli shown. We aimed to strike a balance between providing clarity and keeping the appendix accessible. We would be happy to expand the appendix with additional screenshots from different steps and counterfactuals if the reviewer finds this helpful and more informative. It’s also important to note that our qualitative evaluation is discussed in two dedicated sections, one in the main paper and one in the appendix. In total, we include nine qualitative examples, distributed across both documents, showcasing different models, counterfactual scenarios and step lengths.

What I would personally expect to see in a paper like this is the full user study in a PDF I can read and evaluate. I’m sure you understand why, otherwise it is not possible to evaluate it, as I have no idea what the users saw from start to finish. I don’t believe you even took care to ensure a balanced sample between male/female right? I couldn’t find those details. (but perhaps I am wrong)

Unfortunately these are not possible to fix at this point, but I will adjust my score to account for my misunderstandings.

We sincerely thank the reviewer for their thoughtful and constructive feedback. We truly appreciate the time and effort dedicated to deeply understanding the paper and for providing such detailed comments and suggestions.

Weaknesses:

1. Comparison with TIME and prior white-box methodologies is provided in Table 1. Regarding the omitted citation [1] (Filandrianos et al., AAAI-MAKE 2022), we would like to clarify that it presents the same core algorithm as in [2], written by the same authors, which we already cite and use for direct comparison. We chose to refer to [2] instead, as it includes more extensive experimentation and a broader evaluation across multiple datasets, making it more appropriate for empirical comparison.
   That said, we appreciate the reviewer’s suggestion and agree that including [1] as a reference could provide helpful additional context. We will add it to the related work section of the camera-ready version and briefly explain its relation to the more comprehensive follow-up work of [2].

2. We did not elaborate on the evaluation setup in detail, primarily due to space limitations. As noted in lines 208–210, we follow the exact evaluation methodology described in the related works we compare against ([2] and Dimitriou et al. [3]), to ensure a fair and meaningful comparison. A more detailed explanation of the evaluation procedure will be included in the camera-ready version. Both [2] and [3] rely on semantic counterfactuals driven by knowledge graphs. Their methods retrieve counterfactual images and extract semantic edits that indicate what needs to change in order to shift from the source class to the counterfactual class. However, they do not generate the proposed edits, and thus no minimally altered image is returned. For this reason, we compare the methods based on label flip and number of edits performance on the Visual Genome dataset (Tables 4 and 5).

3. We will revise the parts referenced to improve clarity. If the reviewer could point out specific aspects that were unclear, it would help us focus on our improvements more effectively.

4. We have placed discussion for limitations in the Appendix Line 504. There are certain cases in which artifacts may exist, despite the mitigation procedure,a known problem with diffusion inpainting and generation in general [4]. There are also certain cases in which the classifier might flip the label sooner than human perception, but this is highly variable. We will update the camera ready with a clear limitations section in the Appendix.

Questions:

1. We would like to kindly ask for further clarification regarding what type of adaptation is being suggested. It is not entirely clear to us whether the reviewer is referring to adapting existing methods to operate on new datasets or to modifying their outputs for compatibility with our evaluation pipeline. Below, we provide responses covering both possible interpretations:
   Adaptation of existing methods to other datasets:
   We aim for fair comparisons by evaluating each method in the exact setting proposed by its original authors. We run each baseline using the same dataset, classifier, and data splits as the original work. While adapting each method to a different dataset might seem straightforward in theory, such adaptations often require careful tuning and design choices that may deviate from the original intent. We deliberately avoided introducing such changes to prevent the risk of unfair comparisons.
   Adaptation of methods for human evaluation:
   TIME is included in Table 1 for reference. However, an empirical comparison with human studies is difficult because TIME does not apply semantic edits step-by-step, but instead performs large-scale image manipulations. As a result, it is difficult for human evaluators to identify which change led to the label flip. Additionally, we did not include Conceptual Counterfactuals in the human survey, as it does not generate counterfactual images. Since human evaluation requires visual inspection and judgment, such methods cannot be directly incorporated. If we have misunderstood the reviewer’s concern, we would be grateful for further clarification. We would be happy to provide a more targeted response to ensure that all aspects of the question are fully addressed.

2. We integrate components from [2] and [3]: we base our edits on semantic distances driven by a knowledge graph. Their main limitation is that they do not generate the counterfactual image that corresponds to the semantic edits. This way, not only do they not provide an actual counterfactual image, but by generation it is possible that we need fewer edits to change the class, indicating that both [2] and [3] propose more expensive edit set. Additionally, both [2] and [3] take it for granted that the classifiers under inspection operate based on semantic concepts, without providing any form of evaluation or verification. In contrast, by generating actual counterfactual images and observing whether fewer semantic edits suffice to flip the prediction, we are able to examine and test this assumption explicitly.

3. We will cite the work. Similarly to the previous answer, the method does not generate counterfactual images.

4. A visually incorrect image has generation artifacts or defies commonsense (ie shadows existing without an object to cast it). This characterization is supported from the human survey, in which participants were asked to identify the visually incorrect images during the generation process.

5. We will simplify the figure.

6. The framework aims to identify semantic relationships through well-defined object-oriented semantics. For abstract concepts to be supported, the framework would require a different semantic graph that can provide this information. Since V-CECE is agnostic to the specific source of semantic distances, it can work with any suitable knowledge graph, assuming concept coverage is sufficient. We leave this as future work.

7. In V-CECE, the Editing Module is treated as a black-box within the generative process. While we conduct ablation studies on all other parts of the algorithm, we did not isolate the effect of the masking step specifically. This is because we do not introduce any novelty regarding the parts of the editing module; the novelty lies on the usage of the module to generate counterfactual images, driven by the novel conceptual counterfactual algorithm. Also, this component has already been indirectly evaluated in the original works (GroundingDINO, Segment Anything Model, Stable Diffusion Inpainting). In those studies, the effect of the masking mechanism on the quality and accuracy of the generated outputs has been analyzed as part of the editing system. We rely on these evaluations for the low-level behavior of the generator, focusing instead on the design and evaluation of the full counterfactual explanation framework. If the reviewer considers this ablation important for evaluating V-CECE, we would be happy to include an analysis.

Limitations:

1. We recognize this limitation; however, V-CECE can function on a robust enough semantic graph that contains objects. As long as that graph exists for abstract concepts, the framework can extract information. At this stage, this is left as future work.

2. We would like to clarify that the guarantee of optimality in V-CECE is defined relative to the dataset. V-CECE always returns the shortest semantic edit path within that space, regardless of whether the dataset is biased or sparse. In that sense, the guarantee is not invalidated by dataset limitations. In sparse datasets, the absence of intermediate samples can significantly hinder the construction of interpretable counterfactual transitions. In such cases, concept transitions become coarse, making it harder to capture smooth, interpretable paths. Prior works [2], [3] rely heavily on the dataset structure, and the absence of intermediate examples can lead to unstable or misleading explanations. However, V-CECE directly addresses this limitation by iteratively generating intermediate counterfactuals, one semantic edit at a time, allowing us to densify the semantic space. This gradual construction helps bridge distant concepts through concrete edits, as shown in Table 4, where V-CECE requires fewer steps to reach a decision boundary. As for bias, we believe this is where V-CECE provides a key advantage. In [2], [3], biased co-occurrences in the dataset are reflected directly in the explanations, regardless of whether they are causal for the model’s predictions. V-CECE breaks this dependency by applying and validating each semantic edit in isolation. If a biased concept, e.g a tree, is introduced and does not change the classifier’s output, V-CECE will not mark it as influential. This ensures that dataset bias is not mistaken for model bias. A similar limitation exists in methods like TIME, which learn edit patterns directly from training data.

Paper Formatting:

1. We will update the figure and the caption for clarity .

2. We will correct the spelling.

3. A visually incorrect image has generation artifacts or defies commonsense (ie shadows existing without an object to cast it). We will update the definition in the main paper.

4. Table 3 is cited in line 283-284

[1] Filandrianos et al. Conceptual Edits as Counterfactual Explanations, AAAI-MAKE 2022

[2] Edmund Dervakos et al. Choose your data wisely: A framework for semantic counterfactuals, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

[3] Angeliki Dimitriou et al. Structure your data: Towards semantic graph counterfactuals. Proceedings of the 41st International Conference on Machine Learning,

[4] Cao, Bin, et al. "Synartifact: Classifying and alleviating artifacts in synthetic images via vision-language model." arXiv preprint arXiv:2402.18068 (2024).

We sincerely thank the reviewer for their thoughtful and constructive feedback. We truly appreciate the time and effort dedicated to deeply understanding the paper, which is evident not only in the valuable comments provided but also in the remarkably concise and accurate summary that effectively captures its essence and demonstrates a clear grasp of its core ideas and contributions.

Below, we provide brief responses addressing the points raised under the “Weaknesses” section.

1. We thank the reviewer for this helpful observation. We agree that additional clarification is needed regarding how the diffusion model executes the edits. Due to space constraints, we had originally included the relevant information in Appendix Section B. However, we fully acknowledge its importance for the reader’s understanding. In the camera-ready version, given the extra space provided, we will incorporate a clear and self-contained explanation of the generative process around lines 192–193, previously available only in the appendix. This will ensure that the underlying mechanism is properly conveyed within the main text.

Responses to Questions:

1. The second s_i should be s_j, we will fix it for the camera-ready version.

2. The Stable Diffusion checkpoint utilized for the generative task, as per the description of the model card, has been finetuned for inpainting on laion‑aesthetics v2 5+ , as noted in the model card of the checkpoint on HuggingFace, which does not include images from BDD100K or Visual Genome. The original checkpoint also has been trained on the LAION dataset [1]. Both datasets are from the Common Crawl space, which consists of web page documents and images linked with URLs. As such, the dataset does not include curated datasets, which are invisible to the crawl procedure.

3. The statement in lines 305–308 refers to the DenseNet classifier utilized in prior work and was made with a specific motivation that we now recognize could be explained more clearly. Prior work using the same classifier (DenseNet using BDD100K), such as [2], reports an inability to generate meaningful counterfactual images (Section 4.3, Limitations), attributing this issue to limitations of their algorithm and to the complexity of the dataset. While we acknowledge that these factors may contribute, we argue that the classifier itself also plays a crucial role, which is typically overlooked in previous studies. Through our analysis, we showed that the model’s perception is not aligned with human-level concepts, which significantly limits the potential for meaningful interpretation. In other words, even with a perfect explainability method, the classifier may still fail to produce meaningful outputs, often resulting in outcomes ranging from visually incorrect images to entirely misleading explanations. We will revise the relevant section to articulate this point more clearly and reinforce the reasoning behind our claim.

Once again, we would like to thank the reviewer for their review, and specifically for the time and effort spent to deeply understand our work. Your thoughtful engagement truly means a lot to us and has helped us improve the quality of our work.

[1] Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[2] Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie. Text-to-image models for counterfactual explanations: a black-box approach. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2024.

We sincerely thank the reviewer for their thoughtful and constructive feedback. We truly appreciate the time and effort dedicated to deeply understanding the paper and for providing such detailed comments and suggestions. We are also especially grateful for the kind recognition of the strengths of our work, as well as for highlighting its originality and significance as excellent.

Below, we provide brief responses addressing the points raised under the “Weaknesses” section.

1. Influence of pixel-level distributions in the proposed method is a very interesting point. In our work, we have used the annotations provided by humans using datasets that include this type of information. However, the same method can also operate using an automated system for extracting annotations, such as an object detector. In this case, indeed, there is an influence from pixel distributions. However, this influence is not directly related to the model under inspection, but rather to the auxiliary model that extracts the objects. For the inspected model, only human-level semantics are used, and the biases or pixel-level dependencies are not explicitly passed on. That said, it is correct that the influence cannot be entirely eliminated unless object detectors improve significantly. We will include a relevant discussion in the final version of the paper. Thank you once again for this interesting observation.

2. Uniqueness of the optimal edit sets is another compelling point. Even though uniqueness is not guaranteed (e.g., multiple edits of equal cost may exist), this does not result in more edits, as that would imply a higher overall cost. The determinism of the underlying counterfactual algorithm ensures that ambiguous or redundant edits are avoided. In cases where alternative edit sets exist, the pipeline proceeds as usual, and we have no theoretical or empirical indication that this affects the quality of the explanation. Having two different optimal paths does not lead to degradation, as both adhere to the minimal cost criterion.

3. Diversity of datasets: VG is a highly diverse dataset, covering a wide range of topics and semantics. Other datasets such as Flickr30k and COCO often depict very similar scenes, without object annotations, however, making additional evaluation on them redundant. BDD, on the other hand, covers a distinct real-world scenario not directly represented in VG, which adds valuable generalization insight to our study. Both datasets also give us the ability to compare with prior work.

4. Knowledge graph requirements: We use WordNet (line 109), which indeed follows the required hierarchical structure with “part-of” and “is-a” relations. We have not experimented with other graphs, as WordNet already provides satisfactory semantic distances. Nevertheless, V-CECE is not tied to a specific knowledge graph, any conceptual graph that provides semantic distances could be used. We can make this point more explicit in the final version.

5. Global/non-object features: We acknowledge that the current framework does not capture cases where classifiers rely on unannotated, global, or non-object features such as brightness or texture. However, these are inherently difficult to define in a structured semantic form. For example, brightness is subjective and hard to quantify consistently; even if changed, it does not usually alter the core semantics of the image (e.g., a classroom remains a classroom regardless of lighting conditions). The goal of this work is to explore whether a classifier relies on clearly defined, human-level concepts. Global features fall outside the current scope but present an interesting direction for future work.

Responses to Questions

1. Sensitivity to annotation quality: The proposed V-CECE pipeline does not explicitly rely on annotation quality but, like most ML systems, benefits from high-quality data. Incomplete annotations may lead to seemingly lower-cost edits if essential concepts are missing. Noisy annotations, on the other hand, could result in more costly or misleading counterfactuals. While the pipeline can operate in such scenarios, data quality remains a critical pre-processing factor.

2. Out-of-vocabulary concepts: The datasets used in our study include well-known, everyday concepts already represented in WordNet. If V-CECE is applied to a different domain (e.g., medical imaging), the knowledge graph should be adapted accordingly to ensure appropriate coverage. Since V-CECE is agnostic to the specific source of semantic distances, it can work with any suitable knowledge graph, assuming concept coverage is sufficient and existent.

3. Handling cases like “daytime picture”: Our focus is on well-specified, semantically grounded scenes such as playgrounds, classrooms, and theaters-settings that are easily identifiable by both humans and models. Generic features like “daytime” or “brightness” are not within the current scope. If a classifier is influenced primarily by lighting, that is precisely the type of superficial dependency V-CECE aims to reveal. Style-transfer-like transformations that alter superficial cues without changing semantics could be a future extension to further probe classifier robustness.

Once again, we sincerely thank the reviewer for their time, effort, and thoughtful engagement with our work. Your feedback has been invaluable in helping us improve the quality and clarity of our submission.

We thank the reviewer for engaging in this conversation and for their thoughtful comments.

Answer to Q1:

Conducting an experiment by introducing noisy labels (such as randomly adding or removing nodes according to a particular distribution) is indeed an insightful idea to extend VCECE findings. We have already started setting up the additional experiments; we deem, however, that for the analysis to be relevant, sufficient experimentation would be required. Therefore, we have concluded that the following experiments are necessary:

Consider different noise types: first, randomly deleting bounding boxes. Secondly, substituting bounding box labels with other labels. In the second case, we should consider substituting with semantically similar objects, semantically distant objects or random objects. Therefore, noise type comprises 4 options.

Experiment with the degree of noise: this corresponds to the amount of bounding boxes to be perturbed within the dataset. We can initially experiment with deleting/substituting 5%, 10%, 15% of bounding boxes to evaluate if there are significant changes in VCECE outcomes. Therefore, noise degree adds 3 options.

Validate the statistical significance of these experiments: in each experiment, the choice of which bounding box is going to be deleted/substituted is random. To eliminate any bias or randomness induced by particularly picking certain labels, we run each experiment 3 times.

Overall, to obtain some meaningful results for one model and dataset, we need 4x3x3=36 runs. At the same time, our analysis involves 8 models per dataset.

In our current experimentation, an end-to-end inference run on VG requires ~72h. Therefore, to run 36 experiments of 72 hours each on average on 4 GPUs in parallel needs ~648 hours for one model and dataset, not to mention an evaluation overhead per model. Unfortunately, even for a single model this experimentation surpasses the given deadline for author-reviewer discussion. Therefore, we will be happy to include this analysis in the camera ready.

In any case, we need to mention that annotation quality is not a direct concern of the proposed VCECE pipeline. On the contrary, as in any machine learning application, the user is responsible for the quality of the data given to any module, and the expected results are analogous to this quality.

Answer to Q2: Thank you for raising this important point. Indeed, V-CECE is not restricted to a particular knowledge graph, and we are more than happy to provide additional clarification. The knowledge graph is utilized specifically for determining semantic distances between concepts. For example, given two objects, such as a cat and a dog, the semantic distance can be calculated by finding the minimum distance within the graph. If we consider WordNet as our knowledge graph and assign a cost of "1" to each "is-a” relation, the distance between "cat" and "dog" is computed as "4" due to the four intermediate "is-a” edges.

However, the same computation can be conducted using alternative graphs, such as ConceptNet, provided we appropriately define costs for diverse types of relations, such as "part-of," "HasProperty," and others. By assigning explicit costs to these relations, Dijkstra's algorithm can again be applied to compute semantic distances between concepts regardless of the type of graph. Thus, our method is indeed flexible and not exclusively dependent on WordNet or limited to graphs consisting only of "is-a" type relations. We will clarify this explicitly in the camera-ready version of our manuscript.

Thank you once again for your valuable feedback and suggestions, which greatly assist us in enhancing the quality of our work.