A Mathematical Framework for AI-Human Integration in Work

We thank you for your thoughtful, detailed, and insightful feedback. In response, we added new theoretical and empirical analyses that sharpen our results, test their robustness across modeling choices, and highlight connections to real-world phenomena such as productivity compression.

Please see this PDF for new figures: https://acrobat.adobe.com/id/urn:aaid:sc:eu:fef7d6b2-24f6-4a59-9386-fdf09456ed99.

"Pick a particular empirical result (e.g., Brynjolfsson et al.)... and discuss whether the model explains it."

Thank you for the suggestion. We show that our model formally supports the productivity compression effect observed in Brynjolfsson et al. (2023). Consider two workers from the same ability family (e.g., constant, linear, polynomial), with equal decision-level ability and noise: a low-skilled worker $W_1$ with action-level ability parameter $a_1$ and a high-skilled worker $W_2$ with action-level ability parameter $a_2>a_1$. Let $P_1,P_2$ be their success probabilities before merging with GenAI (whose decision-level ability is always weaker than $W_\ell$ and action-level ability is in the same family with $W_1$ and $W_2$), and $P'_1, P'_2$ after merging. We define productivity compression as: $\mathrm{PC}=(P_2-P_1)-(P'_2-P'_1)$.

Theoretical insight: A corollary of Theorem 3.2 shows that if $a_{AI}>a_1$, then $W_1$’s merged gain $\Delta_1=P'_1-P_1$ can be large-up to $1-2\theta$ for some small $\theta$—while $W_2$'s gain $\Delta_2\approx 0$. Thus, $\mathrm{PC} \approx 1-2\theta$.

Empirical results: Our experiments (Figure 6 in the PDF) show that even when the GenAI ability profile differs from that of the worker, compression increases with the skill gap. For $a_1=0.1$, $a_2=0.8$, compression reaches 0.8, closely matching empirical findings from Brynjolfsson et al.

To our knowledge, this is one of the first formal explanations of the compression effect under realistic assumptions.

"..uniform alternative?"

We chose truncated normal distributions to approximate ability variability observed in empirical benchmarks (e.g., Figure App.9 in Bench authors, 2023). That said, we now include experiments with uniform noise distributions (Figures 3 and 4 in the PDF) and find qualitatively similar results.

"What if you instead took the max error..."

This excellent suggestion led us to study max-based aggregation, where job success probability is given by $P = \prod_{j} \Pr[h_j \leq \tau]$. We conducted both theoretical and empirical analyses:

• Sharper Phase Transition: With identical skills, the transition window scales as $\frac{1}{n}$, sharper than the average-based case $\frac{1}{\sqrt{n}}$.

• Negative Example: We identify a natural setting where no phase transition occurs. With one hard skill (e.g., 0.1) and others easy (e.g., 0.8), success is dominated by the hard one: $P \approx \Pr[h_1 \leq \tau]$, yielding a smooth, non-abrupt transition.

• Empirical Results: Experiments (Figures 1 and 2 in the PDF) confirm these effects. Notably, line-crossing in average-based settings disappears under max aggregation due to its monotonicity.

We will include these results in the final version.

"struggled to map the model into any concrete job..."

Our model is intended to be flexible. For project-based roles (e.g., programming), a job may represent a single project composed of subtasks. For ongoing roles (e.g., teaching or customer service), a job could aggregate performance over a time period. The randomness in subskill outcomes captures intra-personal variability (e.g., fatigue) and task-level heterogeneity. While stylized, we believe this abstraction enables us to reason about human-AI collaboration in both discrete and continuous work environments.

"Prompting GPT-4o plays a key role..."

We will clarify the role of GPT-4o in the main text and include API-based prompting code in the final version to ensure consistency and reproducibility.

“Results are relatively weak and unsurprising.”

While some results may seem intuitive, our theorems precisely characterize when phase transitions in job success occur and quantify non-trivial merging gains—analytically nontrivial and not evident without formal modeling. Our extensions further show when phase transitions disappear (e.g., max-based errors, strong skill dependencies), underscoring the value of our framework.

“Rely heavily on stylized versions of the model (e.g., linear ability functions)...”

We use linear ability functions as an interpretable baseline aligned with prior empirical work (e.g., Brynjolfsson et al., 2023). This choice illustrates key effects clearly and enables tractable analysis. In Appendix C.3, we extend our results to polynomial ability functions, and our experiments confirm that the core insights persist under more complex profiles.

We thank you again for your thoughtful engagement, which has helped us clarify, extend, and strengthen the contributions of our work.

We thank you for your detailed and encouraging review. We are especially grateful for your recognition of our framework’s real-world applicability and for your thoughtful suggestions regarding dataset limitations and empirical grounding, which have directly shaped our additional experiments. Please see this PDF for new figures (https://acrobat.adobe.com/id/urn:aaid:sc:eu:fef7d6b2-24f6-4a59-9386-fdf09456ed99).

"A potential limitation is whether the chosen datasets fully capture the complexity of skill attribution in dynamic work environments. Additional experiments with task-specific benchmarks or real-world AI-assisted work scenarios could further validate the robustness of the proposed approach."

We agree that additional empirical results could further ground our mathematical model and its implications. As one of the first formal frameworks for understanding task-level human-AI collaboration, our empirical results—based on O*NET and Big-Bench Lite—serve primarily as calibration tools to illustrate how the model can be instantiated and to demonstrate that key theoretical insights remain valid in practice. While we recognize the limitations of these datasets—O*NET reflects static, survey-based data, and LLM-based estimates may introduce bias—we have conducted several new empirical analyses (see accompanying PDF) to test the robustness of our findings across alternative modeling choices. We will describe these limitations clearly and include the new results in the final version. We also agree that task-specific benchmarks (e.g., HumanEval for programming, customer support transcripts) could further strengthen empirical grounding and plan to explore such extensions in future work.

• Alternative Error Functions:
  We replace the job/task error aggregation functions $g$ and $f$ with $\max$ (suggested by Reviewer 1zaz), to simulate more fragile task environments. The main patterns remain consistent (Figures 1 and 2 in the PDF linked above), though line-crossings disappear due to monotonicity in the max-based error aggregation.

• Alternative Ability Distributions:
  We substitute truncated normals with uniform noise in ability profiles (Figures 3 and 4 in the PDF linked above), verifying that our key findings hold across distributions.

• Robustness to Task-Skill Graph Variations:
  We randomly modify 5 edges in the task-skill dependency graph (Figures 7 and 8 in the PDF linked above). Despite these changes, the phase transition behavior and heatmaps remain stable.

"It would be useful to discuss potential biases or limitations in these datasets... are there task distributions favoring humans or AI?"

We will explicitly discuss biases in O*NET and Big-Bench Lite datasets regarding task distributions potentially favoring humans or AI. We summarize key observations below:

• Tasks Favoring Humans:
  Tasks that are context-rich and require nuanced judgment, creativity, or interpersonal skills (e.g., strategic planning, ethical decisions) are better captured by human-centered data like O*NET.

• Tasks Favoring AI:
  Structured, repetitive tasks (e.g., basic arithmetic, data classification) are common in benchmarks like Big-Bench Lite, which may overrepresent tasks where AI excels.

These differences suggest that while O*NET may underrepresent emerging digital tasks, Big-Bench Lite might favor tasks with clear, rule-based responses.

"Beyond theoretical modeling, empirical user studies involving real-world AI-human collaboration could strengthen the paper’s conclusions."

We agree this would be valuable, though it is outside the scope of this paper. Our primary goal is to provide a rigorous theoretical framework with proof-of-concept empirical validation. We believe our framework offers a foundation for future empirical and behavioral studies on AI-assisted work. As noted, several of our new experiments aim to move further in that direction.

We thank you again for your thoughtful comments, which have significantly improved our work's clarity and empirical reach. We hope our additions meaningfully address the concerns raised.

Thank you for your thoughtful and constructive feedback. We especially appreciate your recognition of our modeling approach and the suggestion regarding imperfect merging, which we have now incorporated. Please see this PDF for new figures (https://acrobat.adobe.com/id/urn:aaid:sc:eu:fef7d6b2-24f6-4a59-9386-fdf09456ed99).

"...when workers are imperfectly 'merged'."

Thank you for this suggestion and the reference. We extend our merging analysis by introducing a trust parameter $\lambda$ to the merging experiment in Section 4 (Line 358 right column - 439 left column), which models imperfect merging by letting the estimated ability $\hat{c}=\lambda c$ deviate from the true action-level ability $c$ of worker $W_2$. We then assign action-level subskills to $W_2$ when its scaled ability, $\lambda c$, exceeds $W_1$'s ability (i.e., $1 - 0.78s_{j2}\le \lambda c$, extending the setting in Lines 414–415, left column), even though $W_2$ completes skills at level $c$. We analyze the probability gain $\Delta=P_{\text{merge}}-\max\\{P_1,P_2\\}$ across different values of $c$ and $\lambda$ (see Figure 5 in the PDF linked above).

We find that even modest errors in $\lambda$ can sharply reduce $\Delta$. For example, when $\lambda=1.14$ and $c=0.2$, the probability gain becomes $\Delta=-0.2$, indicating that merging reduces job success. This illustrates the critical importance of accurate ability estimation, and complements the findings in Section E.2 on belief-driven merging.

We will create a separate subsection in Appendix E to present these findings clearly.

"...wonder whether another (or a simpler model) would have sufficed."

Our model combines a task-skill graph, subskill-level abilities, and error rate functions to estimate job success (Section 2). We clarify three key points:

• Generality:
  The model allows flexible task-skill dependencies, ability profiles, and error functions. Our framework includes two variants—noise in abilities and subskill division—to capture variation in worker performance. It can be adapted to simpler models while retaining its predictive power.

• Simplicity vs. expressiveness:
  A special case with a single task, noise-free abilities, and no subskill split yields a binary job success probability (0 or 1) and misses correlations across skills. Without subskill division, it is difficult to analyze how merging a human and GenAI tool—each excelling in different subskills—affects performance.

• Non-triviality of results:
  While some conclusions may seem intuitive, formally establishing phase transitions in job success requires careful analysis, as illustrated by the following examples:

  • Dependent skills: By introducing a dependency parameter $p \in [0,1]$, subskill errors are drawn from a shared latent status with probability $p$ and independently otherwise. The phase transition window becomes $\gamma_1=\frac{L\sqrt{\mathrm{sg}(A_1)\cdot\ln\frac{1}{\theta}}}{\mathrm{Infg}(A_1)\cdot\sqrt{1-p}}$, which vanishes as $p \to 1$, showing that stronger dependencies smooth out abrupt transitions.
  • Max aggregation: We show that Theorem 3.1 may not yield a phase transition when using the max operator for error rate functions $g,f$. Consider a job where one critical task is very difficult (e.g., skill difficulty 0.1) while all other tasks are relatively easy (e.g., skill difficulties 0.8). Since the job error rate is defined as $\max_{j\in[n]}h_j$, the overall error is dominated by the hard task. In this case, the job success probability is essentially $P=\Pr[h_1\leq\tau]$, making it a smooth function of the ability on that single, critical task—thus, no sharp phase transition occurs. Notably, the Lipschitz constant here is significantly higher (e.g., $L=1$) compared to the average-case model (e.g., $L=1/(2n)$ in the linear case), which prevents an abrupt transition.

These findings show that richer modeling is necessary to rigorously capture how noise, dependencies, and aggregation affect outcomes like merging. We will include these as theorems in the appendix.

"Modeling jobs as multiple subtasks has been studied..."

Thank you for this pointer. We will cite related works (e.g., Autor and Thompson) and clarify that our contribution lies in offering a formal, ML-grounded framework for modeling skill decomposition and human-AI complementarity. While inspired by similar questions, our analysis tackles fine-grained questions around merging, noise, and error functions not typically addressed in existing models.

"Making the main body.. more high level..."

Thank you. We will revise the presentation to focus on high-level takeaways in the main paper and move derivations to the appendix. We anticipate using the ICML final version page allowance to improve clarity without altering content.

Once again, we thank you for your detailed feedback and for helping us improve the quality and impact of our work.

We thank the reviewer for taking the time to evaluate our submission and for the opportunity to clarify key aspects of our work. Please see this PDF for new figures (link: https://acrobat.adobe.com/id/urn:aaid:sc:eu:fef7d6b2-24f6-4a59-9386-fdf09456ed99).

"..assumptions that skills are independent.,.."

Our model does not assume skill independence. Each skill $s \in [0,1]$ is assigned an ability function, and nearby skills (e.g., 0.7 and 0.8) have similar expected abilities. Thus, correlation across skills is encoded by design. For instance, coding and debugging are correlated due to their proximity in skill space.

The independence assumption applies only to skill execution: once ability functions are fixed, realizations of performance across tasks are treated as independent—assuming task inputs are drawn independently. This is standard in modeling both humans and LLMs. We will clarify this in the final version.

Additionally, Sec. 4 (Lines 373L–356R) already considers dependent skill executions: a parameter $p\in[0,1]$ introduces correlation via a shared latent variable $\beta$. We extend Theorem 3.1 to this setting (details appear in response to Reviewer aCoN).

"deep truths about the job market... but do not make any real claims."

We make no such claim. Rather, we introduce a theoretical framework for analyzing human-AI task-level collaboration, supported by empirical validation. As stated in the abstract and introduction, our contributions include formalizing how job success varies with ability profiles and quantifying the benefit of merging human and AI workers—both supported mathematically and empirically.

We also added new empirical results that may be of interest to practitioners. Notably, we now show that our model captures the productivity compression phenomenon reported in Brynjolfsson et al. (2023). Specifically, Theorem 3.2 implies that merging low-skilled worker $W_1$ with a GenAI tool yields a larger gain in job success probability than merging a high-skilled worker $W_2$, thus narrowing the gap $P_2-P_1$. Since the job success probability correlates with productivity, this explains the empirical narrowing in productivity observed in the cited study. Our heatmaps (Figure 6 in the PDF) empirically reproduce this effect. To our knowledge, our model is among the first to offer a formal explanation of this compression under realistic ability assumptions (details appear in response to 1zaz.)

"No modeling of real labour force dynamics or attempts to test the framework."

Our model is not intended as a macroeconomic tool. It is a task-level, microeconomic framework that enables reasoning about skill decomposition and human-AI integration. It is grounded in real-world data: O*NET (task-skill-job structure), Big-Bench Lite (AI capabilities), and GPT-generated task-skill mappings. Section 4.4 discusses modeling limitations and directions for broader validation.

"I did not check the theorems. I found the assumptions... make the actual model irrelevant."

We respect your decision. However, the theoretical results—such as phase transitions and merging theorems—are central to our work. Understanding when merging yields gains or when success changes abruptly is non-trivial and requires careful analysis. Other reviewers have highlighted the value of this contribution: Reviewer aCoN noted the rigor of the merging results, and Reviewer 1zaz emphasized the model’s potential for interpreting labor studies.

"What am I supposed to gain from the heatmaps?"

The heatmaps illustrate how job success probability $P$ and merging gain $\Delta$ vary with ability profiles:

• Phase transition behavior: The heatmaps confirm whether transitions in $P$ or $\Delta$ occur gradually or abruptly. Distinct color boundaries validate the predicted transitions.

• Extension beyond theory: While Theorem 3.2 assumes identical profiles, Figure 4 shows that phase transitions still emerge when profiles differ.

• Effort required to achieve gains:
  The size of the bright region in Figures 2 and 4(b) indicates how easily a large $\Delta$ can be achieved by merging. Figure 2(b) vs. Figure 4(b) shows that merging identical profiles yields more abrupt gains than merging distinct ones.

These visualizations help connect our analytical results to observable behavior in practical scenarios.

"There is no connection to the real world."

While our model is stylized to allow formal analysis, it is empirically grounded and addresses concrete, real-world questions about human-AI integration at the task level. It uses real datasets (O*NET and Big-Bench Lite) and models skill-action decompositions and merging—challenges at the heart of current research in both ML and labor economics. We hope our clarifications and new results help make this connection clearer.

We thank you again for your time, and hope our responses convey the relevance, rigor, and potential of our work.