Fairshare Data Pricing via Data Valuation for Large Language Models

We thank Reviewer nL1m for their highly constructive feedback and insightful technical questions. The suggestions and feedback by Reviewer nL1m are extremely valuable for this paper. We appreciate the opportunity to clarify these points and address them below.

1. On the Role of Cost cj​ in Fairshare Pricing (Lemma 2):

We thank the reviewer for this sharp and accurate observation. The reviewer is correct that our characterization of the optimal price in Lemma 2 implicitly assumes that a profitable transaction is possible for the seller (i.e., $p_{j}^{*} \geq c_{j}$). We will make this assumption explicit in Section 3.3 to improve the rigor of our claims. We appreciate the reviewer helping us strengthen our presentation.

We would also like to briefly clarify the justification for this assumption. It is grounded in the current dynamics of the LLM data market. Companies are making significant investments to acquire high-quality training data, as its value is a critical driver of model performance. On the other hand, the labor cost to produce this data is often exploitatively low [5-8]. This makes it a reasonable assumption for our model that the optimal price a buyer is willing to pay for valuable data ($p_{j}^{*}$) generally exceeds its annotation and production cost ($c_{j}$).

2. On the Justification of Assumptions 1, 1.1:

We thank the reviewer for this excellent and nuanced question, which gets to the heart of our behavioral model. Our assumption that sellers are sensitive to exploitation is not just an intuition but is strongly grounded in both classical economic theory and direct evidence from the data annotation market.

First, our model aligns with foundational studies in organizational justice and fairness theory, which we cite in our Related Work section. This body of literature shows that worker retention is driven by perceived fairness [1-4], not just absolute payment. When compensation is seen as disconnected from the value created – a core feature of exploitative pricing – it erodes trust and motivation, reducing the willingness of skilled participants to remain in a market. This principle has been documented specifically in the "ghost work" economy, where precarious conditions and low pay lead to high worker churn and disengagement. This has been highlighted in our Related Work (line 104-105).

Second, this long-standing theoretical principle is being validated in real-time in the current LLM data market. Recent investigations by outlets like WIRED [5] and The Washington Post [6] describe the industry as experiencing high worker churn due to persistently low and inconsistent pay. More directly, widespread reports from early 2024 revealed that when platforms like Scale AI's Remotasks abruptly reduced pay rates, it led to an immediate exodus of skilled annotators [7]. Further, a significant 2025 investment in Scale AI by Meta has raised concerns about gig workers being left behind, underscoring ongoing exploitation concerns [8]. These reports provide direct, contemporary evidence for our model's core assumption: exploitative pay strategies lead to high dropout rates among the very workers needed for high-quality data. We will incorporate these contemporary reports into our revised manuscript to further strengthen the real-world grounding for our model's behavioral assumptions.

3. On the Violation of Assumptions 1, 1.1, and 2:

We thank the reviewer for this excellent suggestion. It is important to consider the boundaries of our model. If they were violated, the dynamics would change as follows:

If Assumptions 1 and 1.1 are broken: We thank the reviewer for pointing this out and fully agree on the reviewer’s intuition. This implies that sellers are not sensitive to exploitative pricing and are highly tolerant of being underpaid. In this case, the primary long-term penalty for the buyer (i.e., losing access to high-quality data suppliers) is removed.

If Assumption 2 is broken: This means the buyer is myopic and short-sighted, caring only about maximizing their immediate surplus ($u_t - p_t$) rather than their cumulative long-term utility.

In a scenario where both sets of assumptions are violated, the incentive structure that upholds our fairshare equilibrium might collapses. With tolerant sellers and a short-sighted buyer, the market would be dominated by exploitative pricing, as the buyer would have no rational incentive to pay more for long-term market sustainability. We will add a discussion to Section 3 analyzing how our results would change if these core assumptions were relaxed, making the scope of our framework clearer.

5. Clarification of Figure 2

We thank the reviewer for this careful observation of Figure 2(c). The price point labeled "Buyer 1's MWP" is the highest price at which both buyers are willing to purchase the dataset. Therefore, the seller's total profit at this point is approximately (2 * price) - cost, which is why the profit value (~2.0) can be larger than the price itself (<2.0). If the seller were to increase the price beyond this point, Buyer 1 would drop out, leading to a lower total profit. We will revise the caption of Figure 2 to explicitly state that the simulation involves two buyers and that the plot shows the seller's total profit to ensure this is clear.

4. The Range of Domain:

Thank you for your careful reading. The reviewer is correct. We will update the notation in line 139 from $R\geq0$​ to $R$ for full generality. We thank the reviewer again for their supportive assessment and the detailed feedback, which will improve the rigor and clarity of our paper.

---

[1] Robert G Folger, Robert Folger, and Russell Cropanzano. Organizational justice and human resource management, volume 7. 1998.

[2] Robert Folger and Russell Cropanzano. Fairness theory: Justice as accountability. Advances in organizational justice. 2001.

[3] Jason A Colquitt, Jerald Greenberg, and Cindy P Zapata-Phelan. What is organizational justice? a historical overview. In Handbook of organizational justice. Psychology Press, 2013.

[4] Mladen Adamovic. Organizational justice research: A review, synthesis, and research agenda. European Management Review. 2023.

[5] Niamh Rowe. Millions of Workers Are Training AI Models for Pennies. WIRED. 2023

[6] Rebecca Tan and Regine Cabato. Behind the AI boom, an army of overseas workers in ‘digital sweatshops’. The Washington Post. 2023.

[7] Russell Brandom. Scale AI’s Remotasks platform is dropping whole countries without explanation. Rest of World. 2024

[8] Billy Perrigo. Meta’s $15 Billion Scale AI Deal Could Leave Gig Workers Behind. TIME. 2025.

We thank Reviewer AWaS for their detailed and thoughtful review. We address these points below.

1. Mismatch Between Motivation and Framework:

We thank the reviewer for this insightful observation. It is absolutely correct that the real-world data supply chain often involves a three-party structure: Buyer, Intermediary, and Supplier. Our choice to model a two-party (Buyer-Seller) dynamic is a deliberate one, intended as a crucial first step. Importantly, this model directly represents many real-world markets, such as major crowdsourcing platforms (e.g., Amazon MTurk and Prolific), where buyers interact directly with sellers (annotators).

Our “seller” is a tractable abstraction of the data supply side. The exploitative dynamic we model is often amplified in a three-party reality with an intermediary, a phenomenon explained by the economic literature on intermediation [1,3]. Specifically, the concept of "double marginalization" [2,3] shows how an intermediary's need to secure its own profit margin creates intense downward pressure on annotator wages to satisfy a price-sensitive buyer.

The fundamental conclusion of our work (i.e., under-compensation harms long-term data quality) remains the central tension in both two-party and three-party settings. Our current framework is valuable because it is a tractable model that also directly represents a large fraction of the data market, such as major crowdsourcing platforms. Modeling the additional strategic layer of an intermediary is a crucial and complex issue that demands future research, and we hope our work provides a clear foundation for these investigations.

Our two-party model serves as a tractable and powerful abstraction that isolates this core economic conflict. Formally modeling a three-party game, while a fascinating direction for future work, would introduce significant additional complexities (e.g., contract design, monitoring costs, three-way information asymmetry) that would obscure the foundational insight we aim to establish. We believe it is standard and valuable to first understand the core dyad before modeling the full supply chain.

We will revise the introduction and Section 3 to explicitly acknowledge the three-party structure of the real-world market. We will state that our two-party model is a deliberate abstraction and will briefly introduce the concept of "double marginalization" to argue that our model's core conclusions are likely amplified, not invalidated, in this more complex setting. We will also add a formal three-party game as a key direction for future work in our conclusion.

2. Exit of Buyers/Sellers:

These are excellent points. Even when new sellers can enter the market, the core principle of our work remains the same: underpayment is a self-defeating strategy. This is because, in a transparent market, an exploitative buyer would develop a poor reputation, deterring new, high-quality sellers and leading to a "lemons market" of suppliers. Furthermore, constantly replacing exiting sellers is not a sustainable strategy but a high-cost, inefficient cycle of churn [4]. While formally modeling these entry/exit and reputation dynamics is a key direction for future work, we will add this discussion to our limitations section to scope our contribution and highlight this as an important avenue for research.

3. Change of Data Quality:

We thank the reviewer for these insightful questions. Our framework is intentionally designed to be agnostic to the specific form of the utility function. Thus, it can readily accommodate these complexities, which represent key directions for future work. Fluctuating data quality can be directly modeled by treating the buyer's utility as a stochastic variable; our core conclusions still hold by simply using the expected utility in the decision-making process. We chose a deterministic setup in the paper for clarity and to crisply isolate the core strategic trade-offs, but the framework's ready extension to stochastic cases demonstrates its robustness.

4. On Unclear Figures and Simplified Setups:

We thank the reviewer for this helpful feedback. Figure 1: We will revise it and its caption to better illustrate the role of data valuation and the sequential buyer-seller interaction. Figure 3: This is a simplified illustration to clearly visualize our theoretical result in Lemma 3. We used a fixed utility to isolate the long-term strategic trade-off from stochastic noise. In Section 4, our main experiments adopt stochastic utilities. We will revise Figure 3 to make its illustrative purpose explicit.

5. Complete Information Assumption:

We thank the reviewer for raising this important point. Our choice to model a transparent market, where a platform provides public data valuation scores. This operationalizes information transparency and makes the problem tractable. Our choice to model this setting first was deliberate for three reasons:

1. It serves as an essential first step to isolate the core economic interaction between buyers and sellers before introducing the complexities of asymmetric information.

2. It follows the established precedent of seminal works in economics such as those by Akerlof [4] and Spence [5], which first established key insights in simpler settings. And follow-ups build the pipeline throughout more complex settings.

3. Modeling incomplete information is a vast future research direction, and our work provides the critical first step.

This assumption also aligns with behavioral fairness models that highlight transparency’s role in sustaining cooperation [4,5,7]. We will further clarify in Section 3 that the seller's problem is made tractable by public valuation signals and emphasize how this benchmark lays the groundwork for future study of incomplete information settings.

6. Utility Function Mapping:

We thank the reviewer for excellent questions about utility. Our framework was designed with generality and modularity in mind, allowing it to be adapted to diverse real-world scenarios.

Utility and Commercial Returns: We use “utility” in its classic economic sense as a flexible measure of value [6,7]. Our framework is modular, separating the valuation score $v_k$ (technical impact) from a buyer-defined utility mapping $u_k(v_k)$. This allows practitioners to translate technical impact (i.e., model performance) into firm-specific value, such as commercial returns. This allows our theoretical framework to remain general, while letting practitioners plug in their own specific goals.

Thresholds: The specific mappings in Appendix C are illustrative of this flexibility. Defining empirically-grounded thresholds is a firm-specific implementation detail that our framework is designed to accommodate, leaving this as a direction for future applied work with empirical data.

Combining Valuation Methods: Absolutely agree. The framework is agnostic to how the valuation score $v_k$ is derived. A buyer can use a composite score (e.g., an average or minimum of various methods) for more robust signals, and our conclusions still hold.

We will revise Section 3 to clarify that the utility mapping is a modular, buyer-defined component, highlighting the generality of our framework.

8. On Experimental Setups and Analyses:

We thank the reviewer for this excellent question.

Other utility mappings: In our main experiments, we used the affine mapping (Appendix C.1) primarily for its clarity and direct interpretability. Our primary reason we did not include experiments on all mappings from Appendix C is that our framework is designed to be method-agnostic. The conclusions are expected to hold for other formulations.

To empirically validate this, we run new experiments using the Discrete Outcome mapping (Appendix C.2). The results are highly consistent with those in the main paper: Fairshare pricing continues to maximize long-term utility for buyers and profits for sellers. This confirms that our central conclusions are robust to the specific formulation of utility.

Cumulative Buyer's Utility (Discrete Mapping)

Choice of $\delta$ and $T$: Our selections are grounded in standard practice and validated within our paper.

$\delta$: Following standard approaches in dynamic programming and economics [8,9], we set $\delta=0.98$ to ensure future outcomes are properly considered and not overly discounted. This aligns with our Assumption 2, reflecting forward-looking buyers.

$T$: 100 steps adequately capture the long-term market dynamics central to our analysis, allowing stable patterns and equilibrium outcomes to emerge.

We also thoroughly analyze the impact of these parameters. In Lemma 7 (line 791), we theoretically prove the trade-off, showing that a less patient buyer (a smaller $\delta$) requires a longer time to converge to the optimal fairshare strategy. To complement this, we empirically demonstrate this exact relationship in our robustness check in Appendix E.2.

10. Limitations and discussions:

We thank the reviewer for this important suggestion. While our current limitations section in Conclusion and Appendix G already encourages future work on incomplete information settings, we will add a more thorough discussion in the next version.

---

[1]Coase.The Nature of the Firm.

[2]Spengler.Vertical Integration and Antitrust Policy.

[3]Tirole.The Theory of Industrial Organization.

[4]Akerlof.The market for “lemons”: Quality uncertainty and the market mechanism.

[5]Spence.Job market signaling.

[6]Mas-Colell et al.Microeconomic theory.

[7]Neumann et al.Theory of games and economic behavior.

[8]Sutton et al.Reinforcement Learning: An Introduction.

[9]Stokey et al.Recursive Methods in Economic Dynamics.

We thank 7FBq for the thoughtful review and insightful suggestions. We will address your concerns below.

1. On Shapley-Based Data Valuation Methods:

We thank the reviewer for this valuable suggestion. This provides a welcome opportunity to demonstrate the generality and modularity of our proposed framework. Our "Fairshare" framework is designed to be agnostic (line 62, 347) to the specific data valuation method used. Its core contribution is to model the market dynamics that result from a set of value scores. To empirically demonstrate this, we ran new experiments during the rebuttal period using the Run-in Shapley-based method and following the same setup as Figure 6 in our paper.

For the MedQA on LLama-3.2-1b-Instruct:

As highlighted in these results, the Run-In Shapley method performs similarly with the other data valuation methods. We will include the full suite of results using Run-In Shapley in the next version of our paper.

In addition to this, we will update our Related Work section (Section 2, line 76) to include a thorough discussion of Shapley-based data valuation methods. We will contextualize our chosen methods (e.g., Infl_IP, DataInf) by comparing their scalability and theoretical properties with those of prominent Shapley-based approaches. We will also add this to our Ablation Study discussion (Section 4.3) as a key area for future comparative analysis as these methods become more scalable.

2. On Minor Typos:

Thank you for your careful reading of our paper and for catching these errors. We have identified and will correct all the typos identified by the reviewer (line 17: LLM -> LLMs, line 49: drive -> driving, line 98: thorugh -> through) in the revision. We will perform another thorough proofread of the entire manuscript to ensure correctness.

We thank the reviewer once again for their valuable feedback and strong support for our work. We believe the requested changes will make our paper more comprehensive and impactful.

We sincerely thank Reviewer 9cVB for the thoughtful feedback. We appreciate the opportunity to clarify our core assumptions and contributions.

1. Seller Behavior Assumption and the Role of Cost:

We thank the reviewer for this crucial question, which allows us to elaborate on our model's core mechanics. We would like to clarify that our choice to model seller behavior around the optimal price ($p^{∗}_{j}$) is a justified one for capturing perceived fairness in a transparent market.

$p^{∗}_{j}$​ as a Fairness Anchor: Our framework assumes a transparent market with publicly known data valuation scores (lines 90, 123-125). Here, Maximum Willingness to Pay represents a market-defined anchor reflecting the highest possible value a seller’s dataset $D_j$ can provide to each buyer. Thus, this price also informs sellers about the utility their data contributes, not just a buyer-side metric. A rational seller, aware of this maximum, chooses between accepting "satisficing" [10] on any profitable offer ( $\geq c_j$) and optimizing for a fair share of the value created. Foundational literature in behavioral economics, particularly results from the Ultimatum Game [11], demonstrates that agents frequently reject profitable but deeply unfair offers, choosing to punish unfair behavior even at a cost to themselves. This provides strong empirical support for our core assumption: a seller, perceiving a price well below the $p^{∗}_{j}$ anchor as exploitative, may exit the market. Behavioral models of inequity aversion and reciprocity [12,13] further formalize why fairness significantly influences agents' utility and market participation.

This seller reaction to perceived unfairness is the core mechanism that drives the market to a fair, strategic equilibrium. In dynamic settings, sellers exiting the market serve as credible threats of punishment, aligning with the "Folk Theorem" [7] of repeated games. This theorem suggests cooperative outcomes can be sustained through retaliation. Consequently, it becomes the buyer's own optimal long-term strategy to pay the fairshare price to ensure a stable supply of high-quality data. Our theoretical results in Lemma 3 and empirical findings in Section 4.2 confirm this, showing that the buyer's long-term utility is maximized at the same price that maximizes the seller's profit, creating the mutually beneficial "win-win" outcome that we term "Fairshare".

Integrating Cost as a Foundational Constraint: We thank the reviewer for highlighting the importance of cost. Our model already incorporates it: as shown in Eqns. (3) and (4) (lines 160–163), sellers must achieve non-negative net profit (revenue minus cost).

In particular, under our current model, the conclusion (Lemmas 1-3) hold as long the optimal price $p^{*}_{j}$ is at least as large as the cost $c_j$.

The reason is that the sellers will still lower their willingness or even refuse to participate in the market when receiving compensation below $p^{*}_{j}$, according to our existing assumption. Within this viable market, our framework is built on the foundational assumption that sellers behave rationally, seeking to optimize their own welfare (e.g., profit). This principle of the rational, welfare-maximizing agent is the standard and widely adopted axiom that underpins the majority of modern microeconomic and game-theoretic literature [14,15].

On the other hand, if the optimal price $p^{*}_{j}$ is lower than the cost $c_j$, then it means that either the investment of LLM companies or the marginal utility that the data could contribute is lower than the cost to produce these datasets. And this will lead to a market collapse, meaning that there will be no transactions.

However, in the current norm of LLM data market, we believe that in most cases, the companies are still devoting enormous amount of investment to acquire the data as data is still a very valuable asset, which greatly exceeds the cost (e.g., labor cost) to annotate these datasets [17,18]. And this justifies the implicit assumption such that the optimal price $p^{*}_{j}$ is always larger or equal to the cost $c_j$. We thank the reviewer for pointing that out and will update our writing more rigorously in the next version of the manuscript by explicitly adding the assumption regarding the cost and the optimal price.

Cost-Based Models: Such a model addresses important questions around production viability, specifically, "Is it profitable to produce this data?" Our current framework, on the other hand, focuses on a distinct, yet equally important, economic phenomenon relevant to ongoing debates about fair compensation and ethical data work: "Am I being compensated fairly relative to the value I create?" Recognizing the significance of both approaches, our work serves as a foundational first step, emphasizing value-based compensation as a means to enhance annotator morale and sustain long-term market participation in the LLM industry. We will clarify this perspective explicitly in our revision and highlight the potential for future work to explore cost-based models in parallel with our approach.

2. On the Naming of "Fairshare" Pricing:

We appreciate the feedback regarding the term "Fairshare" pricing. The name was chosen to reflect that our price is not merely seller-optimal, but is a fair equilibrium outcome that benefits both parties in the long run. The fairness of this equilibrium stems directly from our mechanism's core principle: aligning a seller's compensation with the value their data contributes to the buyer's utility, as quantified by data valuation.

As we detailed in our response to the first point, this notion of a sustainable, fair equilibrium is rigorously grounded in both dynamic game theory (e.g., the Folk Theorem [7]) and behavioral economics (e.g., inequity aversion [8,9]). Our theoretical results (Lemma 3) and empirical results (Section 4.2) confirm that this price creates a mutually beneficial "win-win" outcome. The "Fairshare" label captures this crucial alignment of incentives, in contrast to the lose-lose dynamic of exploitative pricing. We will clarify this rationale and its connection to game-theoretic principles in our revision.

3. Federated Learning (FL) Literature:

We thank the reviewer for pointing us to this relevant body of work. We are happy to expand our Related Work section to include more references to this parallel research in our revision, including the ones that the reviewer listed. While both our work and the [Pang et al. 25] tackle data incentivization, they address fundamentally different domains: [Pang et al. 25] focuses on preventing free-riding in decentralized, collaborative settings, whereas our work pioneers a model for a centralized, transactional data marketplace.

4. On the Full Information Assumption:

We thank the reviewer for this critical point. Our choice to begin with a full-information model was a deliberate one, following a long-standing tradition in economic research.

An Essential Benchmark: The full information model serves as an essential benchmark to isolate core strategic interactions and establish a theoretical “best-case” equilibrium. This is crucial for systematically measuring the inefficiencies that arise when more complex forms of information asymmetry are introduced in future work.

Precedent in Economic Research: This approach of starting with a complete information model is a hallmark of seminal economic research, like Akerlof's "markets for lemons" [1] and Spence's "job market signaling" [2]. They both used this methodology to establish groundbreaking insights before the field expanded to tackle various forms of asymmetric information. As one of the first papers to formalize the LLM data market with a game-theoretic framework, we believe providing a tractable model is a valuable starting point.

Scope and Enabling Future Work: Moving to an incomplete information setting is a vast and important future research direction, not a single extension. It requires extensive assumptions about which party holds private information (cost, budget, utility), their prior beliefs, and the signaling mechanisms available. This opens up a vast research program involving different assumptions about private information and agent beliefs. By establishing the core model, we hope our work enables and encourages future research in this area, as we note in our conclusion (line 357-359).

We will revise our limitations and conclusion sections to better frame our contribution as a tractable starting point for this research area and highlight exciting avenues for future work, such as using Bayesian games to model incomplete information.

---

[1] Akerlof. The market for “lemons”: Quality uncertainty and the market mechanism.

[2] Spence. Job market signaling.

[3] Folger et al. Organizational justice and human resource management.

[4] Folger et al. Fairness theory: Justice as accountability.

[5] Colquitt, et al. What is organizational justice? a historical overview.

[6] Adamovic. Organizational justice research: A review, synthesis, and research agenda.

[7] Fudenberg et al. The Folk Theorem in Repeated Games with Discounting or with Incomplete Information.

[8] Fehr et al. A Theory of Fairness, Competition, and Cooperation.

[9] Rabin. Incorporating Fairness into Game Theory and Economics.

[10] Simon. Rational choice and the structure of the environment.

[11] Güth et al. An experimental analysis of ultimatum bargaining.

[12] Bolton et al. ERC: A Theory of Equity, Reciprocity, and Competition.

[13] Dufwenberg et al. A theory of sequential reciprocity.

[14] Mas-Colell et al. Microeconomic theory.

[15] Von Neumann et al. Theory of games and economic behavior.

[16] Brandom. Scale AI’s Remotasks platform is dropping whole countries without explanation.

[17] Perrigo. Meta’s $15 Billion Scale AI Deal Could Leave Gig Workers Behind.

Thank you for the detailed replies! I politely disagree with the author's justification and interpretation of relevant economics research in two core questions.

Artificiality of seller behavior

A rational seller, aware of this maximum, chooses between accepting "satisficing" [10] on any profitable offer ( ) and optimizing for a fair share of the value created. Foundational literature in behavioral economics, particularly results from the Ultimatum Game [11], demonstrates that agents frequently reject profitable but deeply unfair offers, choosing to punish unfair behavior even at a cost to themselves.

This is a great point. I think a model where each seller decides once whether to leave proportional to gap to $p^{\star}_j - p_j$ can be justified using the above literature. However, in the proposed model, the seller leave proportional to $p^{\star}_j - p_j$ at every iteration. This means that for any $p_j < p^{*}_j$, over a sufficiently long time horizon, the probability of the seller staying is 0.

This is amounts to an unfair ultimatum by the sellers - either pay the max price $p^{\star}_j$ or none of us get anything. Note that the buyers have zero utility when paying $p^{\star}_j$. And as the response above points out, such an ultimatum is usually rejected.

Please let me know if I misunderstand the dynamics of the proposed marketplace.

What constitutes a fair price?

Understanding fairness requires understanding how the surplus is distributed amongst the buyers and sellers. Here, all the surplus is taken up by the sellers with buyers having zero utility at the end. Reasonable notions of a "fair" price could equitably split the surplus between the participants, or (better) use the Nash bargaining solution. In fact, there is vast literature in cooperative game theory (including Nash bargaining, Shapley values, envy-freeness) on investigating fariness in surplus division. The current work contradicts such long-accepted notions of fairness in market design. This is why I contest to naming this as a "fairshare" pricing.