Interactive Model Correction with Natural Language

Thank you for your thoughtful feedback. We address your comments below. Please let us know if you have any remaining questions or concerns.

In general, the most significant difficulty I have interpreting the results of the paper is understanding the relationship between the text-based description, in terms of factors like complexity, length, nuance, etc, and the success of this training scheme. The presented results use quite simple descriptions---and it is a good thing such simple texts are useful for getting good results---but how much more complex can these descriptions be? Will CLIP fail to capture the nuance in a marginally longer and more challenging description of an error category? How complex can the text descriptions be for the method to continue to improve results? What are the current constraints here?

Thank you for pointing out this important limitation in the current implementation. Clarify may fail if the backbone vision-language model does not understand the text. CLIP is trained to recognize short captions and is known to be unable to understand longer or more complex descriptions [1]. In our experience, CLIP visually “understands” medium-length phrases with up to two objects: our ImageNet text annotations include “many people in robes,” “rocket in a building,” or “person holding a magnifying glass”. CLIP likely will not understand specialized terms (e.g. “pneumothorax”) or phrases that require multi-step reasoning (e.g. “the color of the heaviest object on the table”).

However, this is not a fundamental limitation of our proposed framework. We expect future multimodal models to be able to understand increasingly more complex descriptions, and we think the bottleneck will ultimately be in efficiently eliciting descriptions from humans, no matter how complex. We note that Clarify automatically bridges from elicited descriptions to model improvements, so any future improvement in the backbone multimodal model brings with it, out of the box, an expanded set of describable failure modes. We have added discussion around this limitation and future work to section 6.

[1] Yuksekgonul et al, "When and why vision-language models behave like bag-of-words models, and what to do about it?."

if the description is not of a common phenomenon, like hair color, will this method fail?

Provided that the description accurately describes a hard subpopulation, the method will only fail if the vision-language model does not understand the text. Our work demonstrates that CLIP is able to understand fine-grained features such as "bleach blonde" and rectify failures due to such features. We believe future multimodal models will be able to understand even more complex descriptions.

I think some efforts should be made to provide guidance and clarity for which types of text descriptions the method can be successful on---this would be quite useful for readers.

In our experience, text descriptions are most successful when they accurately describe a single feature which is not too rare in the general distribution of text on the internet. For example, "bleach blonde" is a good description, but "bleach blonde with a red streak" may be too specific. We believe human-in-the-loop workflows like ours will greatly benefit from future improved multimodal models. We have added this discussion to section 6.

We wanted to follow up to see if the response and revisions address your concerns. We are open to discussion if you have any additional questions or concerns, and if not, we kindly ask you to reevaluate your score. Thank you again for your reviews which helped to improve our paper!

Thanks again for your review. We wanted to follow up again to ensure your concerns are properly addressed. Please let us know if you have additional questions. if all your concerns have been resolved, we would greatly appreciate it if you could reconsider your evaluation of our work.

Thank you for your thoughtful feedback. We address your comments below. Please let us know if you have any remaining questions or concerns.

Currently the method involves re-weighting the training set and re-training the model. Is it possible to used the initially trained model and fix the spurious correlations by further training it?

Interesting suggestion, we tried further training (i.e. fine-tuning) with text feedback from Clarify in the CLIP ResNet-50 setting (Table 1). We show worst-group accuracies on Waterbirds and CelebA below:

Clarify fine-tuning significantly outperforms the naively trained ERM model while showing very similar performance to simply re-training with Clarify. We think this is because the learning objective induced by our text feedback discourages the use of spurious correlations enough that the starting parameters have little effect on the final model.

Comparison with automated methods is not convincing. Table 2 presents comparison with zero-shot/prompted models, while Clarify is a fine-tuning based method. Clarify outperforming RoboShot is not a convincing evidence. Comparison with fine-tuning based spurious correlation mitigation method is needed.

Table 1 compares to representative fine-tuning methods for mitigating spurious correlations, including various variants of ERM, DFR, and Group DRO. The comparison to RoboShot was meant to demonstrate that Clarify is significantly more effective among methods based on text. We agree that this comparison is not apples-to-apples due to the difference in data assumptions; we have moved this comparison to the appendix to avoid giving readers the wrong impression.

What are the design considerations behind the error score in equation 2?

As we are eliciting features that separate correct examples from incorrect ones, we wanted to provide a measure of that. We use class-balanced accuracy to account for scenarios where the number of correct and incorrect examples are different. We have added a bit more discussion to the error score definition.

Paper may be improved from better organization. For example, I don't see a strong motivation that the related work section is between the method and the experiments.

Thank you for the suggestion; we have moved the related work section to right after the introduction. Our original thought was that we may want to jump into new things in this paper (section 4) sooner since our intro and related work are long, but we agree that this organization makes the flow from method to experiments smoother.

(missing references)

Thank you for pointing these out; we were not aware of these and have added them to our related work section.

More information is needed for the baselines and results. For example, how is "class-balanced"/"wort-class"/"slice-balanced"/"worse-slice" defined? Also what is DFR and how is Clarify different from them? Such information is necessary for me to evaluate the contributions of Clarify.

Thank you for pointing out potential points of confusion. We have added the information below to the paper:

• DFR: Deep Feature Reweighting, Kirichenko et al "Last layer re-training is sufficient for robustness to spurious correlations." This differs from the reweighting of Clarify in that DFR requires datapoint-wise group annotations, whereas Clarify only requires one text description of the failure mode.
• class-balanced: ERM loss, balanced so that each class has equal weight.
• worst-class: ERM loss applied to the class with the highest average loss.
• slice-balanced: ERM loss, balanced so that each slice has equal weight.
• worst-slice: ERM loss applied to the slice with the highest average loss per class. Here, "slice" is determined by the annotation from Clarify.

In Figure 8, Clarify seems to help partridge and breastplate categories with a significantly larger gap. Is there anything special about these two categories?

The subpopulations in these two classes have very cleanly visible features, which makes these corresponding failures easier to rectify by reweighting with Clarify.

Sorry if I missed it somewhere, but how is the "7.3% on average" computed?

We identified 31 ImageNet classes with hard subpopulations; the average accuracy on these hard subpopulations is 7.3% higher after reweighting with Clarify.

We wanted to follow up to see if the response and revisions address your concerns. We are open to discussion if you have any additional questions or concerns, and if not, we kindly ask you to reevaluate your score. Thank you again for your reviews which helped to improve our paper!

Thanks again for your review. We wanted to follow up again to ensure your concerns are properly addressed. Please let us know if you have additional questions. if all your concerns have been resolved, we would greatly appreciate it if you could reconsider your evaluation of our work.

CLARIFY shows some keywords as an initial point for further human response. Given the risk with random phrases (getting high error scores) as explained above, I believe the controlled language that the annotators are initially presented with has much role to play than what is emphasized in the paper. Please address related questions to establish their role. What are the keywords presented to the user in Fig. 3 results? Please also present results or your comments on what happened when the user-study is conducted without the initial nudge of keywords?

We do not provide any initial keywords for the user studies in section 5.2 (figures 3, 4, 5). The users came up with all the prompts on their own. Our supplementary video shows exactly what information was provided to these users. We edited the paper to make the setup of the user study clearer. We used automated keyword generation only for the ImageNet experiment, where their primary role was to identify the most promising classes among the 1000.

no single pattern or keyword may explain the many miscalssfied examples.

We agree with the reviewer that there may be multiple patterns underlying model errors, especially in more complex tasks. However, we think this actually points to the value of an interface like Clarify since it can solicit ideas from a diverse set of users to cover all of those patterns that may be relevant and necessary to train robust models.

The top-6 phrases picked on CelebA are all outlier or random features such as darker blonde, darker than..., any other.... Even on WaterBirds dataset in Figure 3, we see somewhat random phrases such as ducks, waterfowl getting good error score. Why is that? On CelebA, darker blonde, dirty blonde resemble outlier features that may have been rare in the training dataset. How can CLARIFY prevent specification of outlier or random features over spurious features?

We believe that outlier features are useful in a similar way to spurious correlations: they identify a specific subpopulation that standard training fails to capture. As we do for spurious correlations, we can adjust the training procedure to account for these failure modes, e.g. by reweighting or collecting more data.

The expert phrases for WaterBird and CelebA are in combination of the label, i.e. forest x water/land bird, man x blonde/non-blonde for WaterBirds and CelebA dataset respectively. CLARIFY only elicits the keyword and not their combination, thereby missing on the true compositional phrase. This is somewhat of a minor issue and goes to only show the difficulty in describing failure modes using language.

Our problem formulation is class-conditional, so it is not necessary to include the class in the description. For example, when we collect phrases for the waterbird class, the phrase "forest" naturally refers to images of waterbirds in a forest.

To your general point, while we agree that CLIP likely struggles to understand descriptions of compositional or otherwise complex failure modes, we do not think this is a fundamental limitation of our proposed framework. We expect future multimodal models to be able to understand increasingly more complex descriptions, and we think the bottleneck will be in efficiently eliciting these descriptions from humans. We note that Clarify automatically bridges from elicited descriptions to model improvements, so any future improvement in the backbone multimodal model brings with it, out of the box, an expanded set of describable failure modes. We have added discussion around this limitation and future work to section 6.

In Table 1, 2 are not well explained. What is LP, DFR, Group Prompt, Class prompt, worst-class, worst-slice? Why is there (ours) marker for some methods? How does CLARIFY work in a zero-shot setting presented in Table 2 because there are no longer examples that can be re-weighted using error description.

Thank you for pointing out this potential point of confusion. We have added the information below to the paper:

• LP: Linear Probing, i.e. training a linear classifier on top of the frozen backbone.
• DFR: Deep Feature Reweighting, Kirichenko et al "Last layer re-training is sufficient for robustness to spurious correlations."
• worst-class: ERM loss applied to the class with the highest average loss.
• worst-slice: ERM loss applied to the slice with the highest average loss per class. Here, "slice" is determined by the annotation from Clarify.
• Group Prompt: Zero-shot prompt based on the group name, i.e. class x spurious feature, from Adila et al
• Class Prompt: Zero-shot prompt based on the class name only, from Adila et al
• (ours): our implementation of the method
• Our method in Table 2 is not zero-shot; it is the same fine-tuning procedure as in Table 1.

Thank you for your thoughtful feedback. We address your comments below. Please let us know if you have any remaining questions or concerns.

Kim and Mo et.al. (Bias-to-Text: Debiasing Unknown Visual Biases through Language Interpretation) that is mentioned in the paper proposed an automated discovery of keywords in the same setting as CLARIFY but without the need for human in the loop. They identified the expert keywords for both CelebA and Waterbirds and even demonstrated some results on ImageNet variants. More elaborate discussion and comparison with Kim and Mo et.al. is expected especially since identifying keywords is not easy for humans.

Indeed, Bias-to-Text is closely related as it extracts keywords using the difference in CLIP embeddings of correct vs incorrect images. We have added an experiment to the paper in which we evaluate Bias-to-Text on the ImageNet classes for which we have annotations through Clarify. The experiment in Table 2 and Appendix B show that both the top 10 keywords and the suggested ranking are not reliable: the top suggested keyword is never the one we identified through Clarify, and ours is only included in their top 10 in 5/31 classes. This suggests that in its current form, automated bias discovery methods ultimately require human oversight to identify the most relevant bias descriptions. We view these two research directions as complementary: automated methods assist human-in-the-loop interfaces, which in turn can guide the design of automated methods. We have added this discussion to our appendix.

Since the error score measures classification accuracy between two classes, the error score using random similarity values must be around 0.5. Yet, the error scores reported for genuine descriptions reported in Figure 3 are often less than 0.5. Since error score is intended to guide the user, how is the annotator expected to interpret the worse the random error scores?

Thanks for catching this; there was a difference between our definition in eq (2) and our implementation: the error score is calculated as (best_acc - 0.5) * 2, which is normalized so that it ranges from zero to one. Random similarity values would have an error score of 0, not 0.5. All prompts from the user study do substantially better than random similarity values. We have fixed the description in the paper and made it clear that random similarities achieve zero error score.

The experiments did not paint a convincing role of humans in specifying the error pattern. Humans are not necessarily good at finding common patterns across many misclassified examples. Please justify the required human skill and the potentially poor payoff for their effort (i.e. performance payoff for each keyword may not worth the effort).

While we agree that finding patterns in misclassified examples is non-trivial, our user study shows that we can effectively operationalize this task. Specifically, given an interface juxtaposing two grids of correct and incorrect images side-by-side, non-experts with no prior experience could identify failure modes at an average pace of 2.7 minutes per dataset. Prior works in the HCI literature show that crowds of people excel at generating ideas [1,2], particularly when asked to compare and contrast two cases to extract high-level differences [3,4].

We believe that when identifying failure modes, including humans in the loop is crucial. First, we note that even automated methods ultimately require human oversight to validate that identified keywords genuinely represent significant subpopulations. Our research recognizes this dependency and offers an efficient framework for eliciting these descriptions from humans. Second, as our user study demonstrates, non-expert users are able to identify features that are uncommon phrases or are otherwise not easily detected, such as “bleach blonde” or “dirty blonde.” Finally, even if a given keyword doesn’t end up improving performance in an immediately measurable way, it provides useful feedback to the model developer as a fine-grained audit/evaluation of model behavior that isn’t possible through standard validation sets. This can result in efforts to monitor the performance of error-prone subgroups or even just help the model developer be aware of the non-resolvable limitations of the model.

[1] Yu and Nickerson, Cooks or cobblers? crowd creativity through combination (CHI 2011).

[2] Kim, Cheng, and Bernstein, Ensemble: Exploring Complementary Strengths of Leaders and Crowds in Creative Collaboration (CSCW 2014)

[3] Fogarty et al, CueFlik: interactive concept learning in image search (CHI 2008).

[4] Cheng and Bernstein, Crowd-Machine Learning Classifiers (CSCW 2015).

We wanted to follow up to see if the response and revisions address your concerns. We are open to discussion if you have any additional questions or concerns, and if not, we kindly ask you to reevaluate your score. Thank you again for your reviews which helped to improve our paper!

Thanks for the response and additional experiments comparing with Bias-to-text. Some of my concerns are resolved, but my major one is still not.

I did not find a description of Subgroup Acc in Table 2, could you please clarify? I also do not understand how the phrase for "ours" is picked (from multiple human eliciations).

In response to my comment that the keywords picked by humans look like they are outlier or random, the authors responded as "We believe that outlier features are useful in a similar way to spurious correlations: they identify a specific subpopulation that standard training fails to capture. ". Yet, Appendix B and Table 2 shot down Bias-to-Text for the same reason that it picked many outlier features. Besides, there can be multiple spurious features and we cannot just evaluate Bias-to-Text based on overlap with the spurious feature identified by CLARIFY. Replication of Fig 7,8 using the keywords picked by Bias-to-Text would have made for a more convincing argument.

Also, what happens when we use the scorer of CLARIFY for Bias-to-Text or the scorer of Bias-to-Text in CLARIFY? Because it seems like a major contribution of CLARIFY is the scoring measure.

I am still not convinced that a human can be expected to unearth spurious features. The selection of correctly classified and negatively classified examples is bound to influence the perception of the annotator. How does CLARIFY operate when there are many errors or many spurious patterns?