Language Agents for Detecting Implicit Stereotypes in Text-to-image Models at Scale

Weaknesses:

1. We acknowledge the difficulty in fully capturing the diversity and complexity of stereotypes in real-life scenarios. In our study, we attempted to increase the diversity of evaluated samples by drawing from toxic text content datasets on various social platforms. Regarding the imbalanced distribution of subgroups, we would like to clarify that Figure 3 illustrates the proportional distribution of instruction pairs obtained during the initial text extraction process, which is biased towards specific subgroups. It is important to note that this distribution does not represent the training data. Instead, it highlights the strong stereotypes associated with certain subgroups in everyday expressions.
2. In the Appendix section, we provide a comprehensive evaluation of the proposed agent framework, including an analysis of false positives, false negatives, and the impact of different parameters. We apologize for not making this clear in the main text and will revise accordingly.
3. We compare our proposed agent framework with existing stereotype detection methods. It's important to highlight that these conventional methods rely on manually crafted prompts for comprehensive testing.

4. Our agent framework emphasizes the use of existing models to complete complex tasks. We add a more comprehensive justification for the selection of specific tools.

5. We understand the importance of discussing the potential impacts and implications of stereotype detection in practice. In the revised paper, we provide a more thorough discussion of the unintended consequences of bias mitigation strategies, the role of human judgment in determining stereotypes, and the balance between freedom of expression and risk mitigation.

6. In response to the concern about providing explicit recommendations or strategies for mitigating biases, we include a section discussing potential bias mitigation approaches and their implications in the revised paper.

Questions:

1. Language agents for stereotype detection in text-to-image models may face limitations in accurately detecting subtle stereotypes or those embedded in complex contexts. Additionally, they may struggle with cases where stereotypes are expressed using sarcasm or irony. We discuss these challenges and potential solutions in the revised paper.
2. For manual annotation, annotators were provided with guidelines that included descriptions and examples of stereotypes. They were instructed to determine the presence of stereotypes based on these guidelines. We provide more information about the criteria used for manual annotation in the revised paper.
3. We apologize for the lack of detail regarding the annotation process. In our study, multiple annotators were involved, and we conducted an inter-rater reliability assessment to ensure consistency. We include this information in the revised paper.
4. The toxic text datasets used to construct the benchmark dataset were selected based on their relevance to the task, diversity of content, and representation of various social platforms. We provide more details about the selection criteria and guidelines in the revised paper.

[1] Jaemin Cho, Abhay Zala, and Mohit Bansal. 2022. DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models. https: //doi.org/10.48550/ARXIV.2202.04053.

[2] Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. 2022. Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale. https://doi.org/10.48550/ARXIV.2211.03759

[3] Naik R, Nushi B. Social Biases through the Text-to-Image Generation Lens[J]. arXiv preprint arXiv:2304.06034, 2023.

Regarding the technical novelty of our paper, we understand your point about the use of existing models and metrics. However, our primary contribution lies not in the novelty of the models or metrics, but in the innovative application of these tools. Our core motivation is to build a language agent that can perform task planning and effectively utilize various tools to detect and identify a broader range of stereotypes present in social media texts. Therefore, this agent overcomes the limitations of previous work that required exhaustive testing of all possible occupations and other descriptors to construct prompts.

We appreciate your concerns regarding open sourcing the code of our study. While we have included the code in the submission auxiliary materials, we understand the need for it to be easily accessible. Therefore, we commit to open sourcing the code in an easy-to-access format in subsequent papers. We will also provide ample examples to illustrate the seriousness of stereotypes in existing text-to-image models, further aiding in the understanding and reproduction of our work.

1. Weaknesses:

W1a: We acknowledge your concerns regarding the framing of our method as an autonomous agent and agree that it might have been overly ambitious. While you pointed out the dependence on "human intervention," it is crucial to note that, at the present stage, crafting well-written prompts manually remains a vital component in enhancing the performance of an agent. Our intention was to highlight the potential for autonomy rather than present a fully autonomous system. In light of your comments, we will revise our framing to better represent the current state of the method and potential future developments.

W2a: We compare our proposed agent framework with existing stereotype detection methods. It's important to highlight that these conventional methods rely on manually crafted prompts for comprehensive testing.

W3a: We agree with your observation that the method is better explained as automated red-teaming rather than benchmarking. We will revise our framing accordingly to ensure that the discovered issues are highlighted as important aspects of red-teaming, rather than focusing on consistent benchmarking.

1. Questions: In response to your questions, we would like to provide the following information:

• Diversity of stereotypes: Our experiments uncovered more than 2800 prompts that simultaneously showed stereotypes pointing to one subgroup across multiple models. These prompts cover the special expression of "verb + noun" that was missing in previous studies, such as "commit sadistic murders" and so on.
• Autonomy and flexibility of the agent: We acknowledge that our method currently involves human intervention and custom steps. However, our goal is to progressively refine these processes towards greater autonomy and flexibility for new tasks.

Regarding the diversity of images, we would like to clarify that all prompts start with a fixed "The people who ___" to avoid the prefix prompt introducing additional stereotypes. This approach ensures that the uncovered stereotypes are solely attributed to the models and not influenced by the prompts themselves.

[1] Jaemin Cho, Abhay Zala, and Mohit Bansal. 2022. DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models. https: //doi.org/10.48550/ARXIV.2202.04053.

[2] Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. 2022. Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale. https://doi.org/10.48550/ARXIV.2211.03759.

[3] Naik R, Nushi B. Social Biases through the Text-to-Image Generation Lens[J]. arXiv preprint arXiv:2304.06034, 2023.