Random Graph Asymptotics for Treatment Effect Estimation in Two-Sided Markets

Dear Reviewer nxGk,

Thank you for your thoughtful review of our work. We appreciate your recognition of the practical and academic significance of addressing treatment effect estimation under interference in two-sided markets, as well as your acknowledgment of the theoretical soundness and empirical validation of our framework. Below, we address your concerns in detail.

Q1. Contribution to the Causal Reasoning Community

While our framework builds on the foundational work of Li and Wager’s, our contributions extend their methods to the unique and important setting of two-sided markets. We model the interference on both sides of the market (listings and customers) under two-sided randomization, introducing a more complex randomization and interference structure than the single-sided setting considered in Li and Wager’s. Specifically, extending Li and Wager’s results to our context required addressing two major challenges:

• Incorporating a random graph model in the two-sided randomization setting. Li and Wager’s results are limited to a single set of treated units. Initially, we considered including both the listing side and the customer side in the random graph model since both sides receive treatments and controls. However, upon carefully analyzing the mechanics, we concluded that the interference between customers is more appropriately modeled within the potential outcome function (Eq. 2) as a function of both graph topology and equilibrium dynamics, and the interference between listings is best captured through the random graph model.

• Determining the specific form of the graphon. Unlike the static setting in Li and Wager’s work, our framework necessitates a dynamic interference model. To address this, we focused on the steady-state environment. The graphon represents the probability of two listings being connected, accounting for the interference between them. Since interference only occurs among listings available at the same time, we use the steady-state mass of listings to determine the graphon value in Eq. 1. Our work incorporates equilibrium dynamics, which are critical for two-sided markets but absent in Li and Wager’s. This required adapting their methodology to account for steady-state circumstance, as well as deriving new variance adjustments for treatment effect estimation.

We revised the “Our Contributions” section to better highlight these distinctions, technical challenges and the methodological decisions they required in extending Li and Wager’s framework to the two-sided market context.

Q2. Concerns About Similarities

We appreciate your concern about similarities with Li and Wager’s work. Our paper explicitly acknowledges Li and Wager’s as a key reference. We cite their paper properly multiple times, made our own contribution clear and properly cites their contributions throughout. While our work builds on their framework, we have introduced significant methodological advancements, as detailed in the previous section. Some overlap in text arises from our adaptation of their assumptions and random graph model, which is common in academic writing when building on existing frameworks. Similarities in figures stem from using the same criteria to assess the estimator, but the code for generating our figures is original and available in the supplementary materials. While some formulas share structural similarities with those in Li and Wager’s work, they have been tailored to suit the two-sided market context. The revision of our manuscript explicitly highlights our methodological novelties and ensures that our contributions are distinct, clearly stated, and appropriately credited.

Thank you again for your constructive feedback. Your comments have been invaluable in identifying areas for improvement. Please let us know if there are any additional points you would like us to address.

Dear Reviewer d4Pv,

Thank you for your thorough review and insightful feedback on our submission. We greatly appreciate your recognition of the rigor of our mathematical framework, the novel use of graphon models, and the computational tractability of the proposed methods. Your comments have provided valuable insights and highlighted potential extensions to our work.

Q1. Adaptation to Observational Settings

Your suggestion to extend our framework to observational settings is highly valuable and aligns with potential directions for future research. While our current method is tailored for randomized experiments, adapting it for observational data represents a natural extension of our work. Propensity score methods and covariate adjustments could be integrated to account for confounding; potential methods such as the use of proxy variables could address selection bias and unobserved confounders; matching similar time periods and controlling for external factors could improve the validity of observational estimates. We indicated these possibilities in the conclusion section as future research directions in the revised manuscript.

Q2. A/B Testing Framework

We appreciate your suggestions regarding the design of an A/B testing framework. While this lies beyond the scope of our current study, your proposed ideas, such as simplified treatments, gradual rollouts, and tracking spillover effects, provide valuable guidance for translating our framework into practical applications. We also highlighted these points as future research directions in the revised manuscript.

Q3. Practical Applicability of the Method

We acknowledge that real-world markets exhibit complexities such as seasonality, variable durations, and heterogeneous attributes. However, the assumptions in our framework, including Poisson arrivals, multinomial logit choice, and exponential occupancy times, serve as well-established and tractable approximations that allow us to focus on the core dynamics of interference and treatment effects. These simplifications are widely used in the literature to provide a theoretical foundation, which can be iteratively refined in future work. Extensions to incorporate time-varying patterns or more flexible models are promising future directions that build on our current framework without altering its core contributions.

Once again, we thank you for your constructive feedback. While the suggestions you provided represent valuable directions for future research, they require substantial modifications that are beyond the current scope of this submission. We look forward to exploring them in future work. Please feel free to let us know if additional clarifications are needed.

Dear Reviewer zKRQ,

Thank you very much for your thoughtful and detailed review of our work. We deeply appreciate the time and effort you took to provide constructive feedback, which has been instrumental in helping us refine and clarify our contributions. We are pleased that you found our modeling of listing dynamics and equilibrium effects practical and relevant for real-world applications. Your acknowledgment of the significance of studying treatment effects in two-sided markets further motivates us to enhance the quality of our work.

Q1. Technical Challenges in Extending Li and Wager’s Results

Extending Li and Wager’s results to our setting required addressing two major challenges:

• Incorporating the random graph model in the two-sided randomization setting. Li and Wager’s results are limited to a single set of treated units. Initially, we considered including both the listing side and the customer side in the random graph model since both sides receive treatments and controls. However, upon carefully analyzing the mechanics, we concluded that the interference between customers is more appropriately modeled within the potential outcome function (Eq. 2) as a function of both graph topology and equilibrium dynamics, and the interference between listings is best captured through the random graph model.

• Determining the specific form of the graphon. Unlike the static setting in Li and Wager’s work, our framework necessitates a dynamic interference model. To address this, we focused on the steady-state environment. The graphon represents the probability of two listings being connected, accounting for the interference between them. Since interference only occurs among listings available at the same time, we use the steady-state mass of listings to determine the graphon value in Eq. 1.

We revised the “Our Contributions” section to better highlight these challenges and the methodological decisions they required.

Q2. Constant Graphon Function in Eq. 1

You are correct that using a constant graphon simplifies the interference structure, and a more nuanced model with graphon values determined by listing types is a promising direction for future work. However, as we discussed in the last question, interference effects are most influenced by whether a listing is available rather than its specific type. Thus, our choice of a constant graphon suffices to derive interpretable and tractable estimators under equilibrium dynamics.

Q3. Line-by-Line Issues

We have revised the manuscript to address the following formatting and clarity issues:

Line 068 (former line 66): Corrected the citation format.

Line 179 (former line 167): Repeated the definition of Y_i for clarity.

Line 227 (former line 225): Used distinct indices for the treatment conditions of listings and buyers.

Line 379 (former line 380): Replaced “regularization assumptions” with “regularity assumption.”

These changes improve the consistency and readability of the manuscript, and we sincerely thank you for carefully reading it and pointing them out.

Q4. Explanation of Eq. 2

We agree that Eq. 2 could be introduced more clearly. The form of the potential outcome function is not an assumption but is derived, as detailed in Appendix A.1 of our revised manuscript. To improve clarity, we explicitly define it as the steady-state booking rate. However, while the potential outcome function is derived from the equilibrium described in Section 2, its specific form as a function of the fraction of treated neighbors relies on Assumption 3, and its simplification also depends on Assumption 2. For this reason, we can’t place it earlier in the manuscript.

Dear Reviewer 8dZA,

Thank you very much for your constructive review of our work. We greatly appreciate your recognition of the challenges posed by network interference in two-sided markets and your acknowledgment of the novelty of our proposed method utilizing random graph asymptotics. Additionally, we are glad that the practical applicability and effectiveness of our method, as demonstrated through numerical simulations, resonated with you. Your feedback has provided valuable insights that will help us improve the clarity and comprehensiveness of our paper.

Q1. Comparison with Existing Methods

• We agree that a more detailed comparison with existing methods would better highlight the advantages of our approach. In the “Related Works” section of our manuscript (lines 59–70), we indicate that in the existing methods, their estimators are particularly sensitive to the relative volumes of supply and demand, leading to biases due to the interconnected nature of participants. Furthermore, these methods lack desirable asymptotic properties and only provide point estimates without confidence intervals.

• In contrast, our approach addresses these limitations by not only offering confidence intervals but also demonstrating robust asymptotic properties. Our estimator for the direct effect is asymptotically normal around the true direct effect in the presence of interference, and we quantify the excess variance introduced by interference effects. The PC-balancing estimator for the indirect effect is both consistent and asymptotically normal.

• In our revised manuscript, we updated the “Our Contributions” section to provide a clearer understanding of our method's contributions and its position relative to existing work.

Q2. Comparative Experiments with Existing Methods

• We also agree that adding comparative experiments would strengthen our paper. In the Appendix A.3 of the revised manuscript, we include comparisons of our estimator with the customer-side estimator, listing-side estimator, and TSR-Naïve estimator proposed in [1].

Once again, we sincerely thank you for your thoughtful comments and encouragement. Please do not hesitate to reach out if further clarifications are needed.

---

[1] Ramesh Johari, Hannah Li, Inessa Liskovich, and Gabriel Y. Weintraub. Experimental design in two-sided platforms: An analysis of bias. Management Science, 2022.