A Conceptual Framework for Analyzing Social Representation in Unstructured Data

1. Inconsistent framing of context/use case: The abstract frames the paper in the context of "foundation model development" but fails to make this distinction throughout the paper.
2. Too vague, to the point of limited utility: throughout the paper, the authors assert that specific measures cannot be delineated. For example, they write that social characteristics "measurement must be adapted to local context and time" and because they vary, "static identity term list cannot be exhaustive." The authors also do not make a distinction between different modalities of data, e.g. audio vs. visual vs. text, which have very different ways of having traces of social data that are worthy of being discussed. Yet this vagueness seems directly at odds with the authors' goal of developing a framework that is easily applicable. The paper suggests that it serves as a "starting point" (Section 5), but this seems insufficient given that most of these ideas are from other existing works. 2.1. In Section 3.1.1, the suggestion of "faces or bodies in visual data or voice in audio data" are good suggestions but do not come close to being thorough in the range of possibilities, which makes it limited in utility. For example, https://arxiv.org/pdf/2309.15084.pdf (Figure 2 on page 4) is a good example of a more thorough survey of different types of human data that occur specifically in visual data. A similar type of typology would strengthen this work.
3. In "Social Characteristics," the authors use the primary examples of "perceived" social characteristics that "must be inferred, frequently using predictive methods." This sidelines the many ethical issues around predicting social characteristics and reproduces these harms: see a) Lockhart, Jeffrey W., Molly M. King, and Christin Munsch. "Name-based demographic inference and the unequal distribution of misrecognition." Nature Human Behaviour (2023): 1-12. b) Cao, Yang Trista, and Hal Daumé III. "Toward gender-inclusive coreference resolution." arXiv preprint arXiv:1910.13913 (2019). c) So, Richard Jean, and Edwin Roland. "Race and distant reading." PMLA 135, no. 1 (2020): 59-73. d) Hanna, Alex, Emily Denton, Andrew Smart, and Jamila Smith-Loud. "Towards a critical race methodology in algorithmic fairness." In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp. 501-512. 2020. Given that social representations of people are a central focus of this work, such ethical concerns should be much more centered in this paper.
4. Given the broad utility that the authors aspire toward, the case studies are not particularly convincing of how the framework is easily or widely applicable.
5. The example of evaluating age representation in C4 and queer representation in LAION-400M is interesting, but seems to presuppose a lot of existing knowledge about what concerns/harms the authors are looking for.

• Lacks empirical evidence on the framework's risk mitigation effectiveness for downstream tasks.
• Absence of comparative analysis with other methods tackling similar issues.
• Despite acknowledging the variability of "good" social representation, the framework does not adapt to different contexts or provide standard risk measurement metrics.
• There is no demonstration of the framework's impact on risk perception before and after its application.
• While the framework poses high-level, human-centered questions, it falls short of offering standardized metrics for practitioners to use for development and iterative assessment.

• The paper is difficult to read. The writing style and the structure can be improved. The big picture and the aim of the framework are discussed a lot, but there are not a lot of details discussed. Figure 1 needs to be more detailed on the actual components of the framework.
• The novelty in the methodology is limited. It is not clear what are the newly proposed ideas and methodology in the proposed framework.
• This paper might be a better fit for a more applied track and a conference/workshop/journal with a focus on such applications/frameworks.
• The experimental setup is not very clear. It would be easy to follow if the authors can add the dataset details in a table, provide with the research questions and a quantitative way to evaluate the results.
• The proposed framework is more discussed in the paper as a hypothesis and how it will work theoretically, than in practice showing how it works according with results.

• The paper primarily focuses on qualitative assessments of data and provides limited quantitative metrics. Quantitative measures are important in enhancing the rigor of dataset evaluation and making the framework more comprehensive. Focusing on qualitative analysis makes the conceptual framework, as presented, time-consuming to apply comprehensively, especially to large datasets. This suggests further refinement is needed to make it more scalable for practical use.
• The paper lacks empirical validation of the framework on a diverse set of datasets and applications, which are necessary to demonstrate the modality-agnostic nature of the framework. Demonstrating its effectiveness through real-world case studies, showcasing how easy it is to extend to extensible to new modalities and combinations, and providing comparative analysis with similar frameworks would strengthen the contributions.
• While the paper identifies issues in datasets, it offers limited discussion on effective mitigation strategies. For example, there is a risk of introducing unintended bias when applying the framework, as the choices made in the analysis and actions can themselves be influenced by the researchers' perspectives. It is unclear how practitioner decisions affect the insights stemming from the application of this framework.
• Technical contributions are limited. The conceptualization of a general framework for social identity representation under three dimensions (Who, What, Human × Content) has merit and the potential to serve as foundations for researchers and practitioners working on responsible AI, but the focus on generality across datasets, modalities, and use cases, make the proposed work vulnerable to oversimplification. Different datasets and applications may require nuanced and context-specific approaches, which the framework might not fully capture. It would be beneficial to provide more specific guidelines or techniques for conducting the analyses and actions proposed in the framework.

The work could benefit from further refinement, the inclusion of quantitative analysis to facilitate scalability, and more detailed guidance on mitigation strategies, in particular addressing potential bias in the framework's application.

1. Methodological Development: The methodology behind the framework lacks depth, particularly regarding its applicability to the wide variety of unstructured data that exists. This concern is especially relevant given the increasing complexity and scale of datasets to be utilized in deep learning models.

2. Validation and Empirical Evidence: The primary weakness of the paper lies in its validation process. The utilization of toy examples, while helpful for illustrative purposes, does not suffice to establish the efficacy of the proposed framework across the varied landscape of unstructured data that practitioners face. For robust validation, the framework should be subjected to a battery of tests across datasets of different scales, complexity, and real-world messiness. This would not only reinforce the reliability of the framework but also demonstrate its practical applicability and adaptability. Moreover, the current lack of quantitative analysis means there is no clear measure of performance improvement or bias mitigation when using the framework, which is crucial for empirical validation. This rigorous level of empirical evidence is necessary to validate any claims of improved fairness and efficacy.

3. Comparative Analysis: A significant drawback of the paper is the absence of a detailed comparative analysis with current state-of-the-art frameworks and methodologies. Such a comparison is vital to critically evaluate the advantages, limitations, and added value of the new framework. The paper should articulate how and why the proposed framework outperforms existing methods or, alternatively, how it could be integrated with current approaches to provide a more comprehensive tool. Specific benchmarks, metrics, and scenarios where the proposed framework excels (or falls short) compared to others would greatly enhance its credibility and relevance.

4. Actionability and Implementation Guidance: The action steps are conceptually insightful but lack the necessary detail for practical implementation. For practitioners to adopt the framework, they require concrete guidance on its integration into existing workflows. This should include specific recommendations on handling different types of unstructured data, along with examples of the framework in action. Case studies that demonstrate successful integration and outcomes are needed to bridge the gap between theory and practice.

5. Impact Measurement: The manuscript does not adequately address how to quantitatively measure the impact of the framework on AI fairness and bias mitigation. Presenting specific metrics and assessment methodologies is essential for validating the framework's effectiveness in practical applications.

.