Multi-Session Budget Optimization for Forward Auction-based Federated Learning

Thank you for your insightful and encouraging feedback. Below, we provide detailed, point-by-point responses to the key questions raised in the comments.

W1. More motivation of why multi-session auction scenario is realistic seems to be helpful.

Practical federated learning environments typically require sequential collaboration phases where the collective model evolves gradually through repeated information consolidation and parameter adjustments. Consider healthcare applications: medical facilities (serving as information contributors) supply clinical data incrementally across various timeframes, leading to progressive enhancement of the shared analytical framework.

The operational reality demonstrates that participating entities join the collaboration asynchronously, while available funding resources may vary throughout the project lifecycle. This dynamic creates an ideal application for multi-round bidding structures. Such an approach accurately reflects the continuous nature of knowledge development in these collaborative ecosystems, where both informational assets and processing capabilities are distributed through successive competitive allocation events.

W2. The paper would benefit from a clearer exposition of its assumptions regarding data owner behaviors. While the authors describe the decision decision-making process of data consumers, it's unclear whether these are based on empirical observations, theoretical models, or simplifying assumptions. A more explicit statement of the behavioral assumptions underpinning the model would strengthen the paper's foundation and help readers better understand the scope and limitations of the proposed approach.

The method we've developed operates without assumptions regarding the other types of stakeholders' behaviors. Rather, it features inherent adaptability to accommodate diverse stakeholder actions across multiple contexts. Our outlined decision protocols maintain flexibility and remain independent of underlying models, enabling effective functioning regardless of participant motivations or strategic choices. This versatility represents a fundamental advantage of our approach, as it facilitates widespread implementation across various aggregated federated learning environments without limitations imposed by strict behavioral assumptions or predefined conduct expectations.

W3. It is suggested to discuss the technical challenges of designing proposed mechanisms.

The first technical challenge lies in optimizing the computational efficiency of utility estimation. Our methodology implements a framework combining Shapley-derived valuations with beta-based reputation tracking to quantify the utility a DC can gain from each DO. However, while Shapley-based methods are conceptually straightforward, they are computationally intensive. Additionally, another challenge is ensuring robust winning price modeling across diverse data owner distributions, as well as balancing the bidding strategy’s performance with budget constraints. Addressing these complexities is crucial for adapting our approach to practical AFL market environments, and we will add further details on these points in the revised paper.

Typo

Thank you for pointing that out. We have corrected the typo in the revised manuscript.

What bidding strategies did the other bidders use in the experiments? Were their approaches similar or distinct?

As detailed in the experiment section, we compared diverse advanced bidding strategies such as FBs, FBc, among others, by various bidders. This setting is designed to address concerns regarding the potential impact of competitors with advanced bidding strategies. However, if all bidders adopt the same bidding strategy, the differences in their individual historical data could lead to variations in their utility estimation and winning price models. This variance might result in differences in their bidding behaviors. Moreover, in practice, different bidders target diverse data owners due to their distinct training tasks. These factors influence their bids through specific parameters and model approaches in their bidding strategies. Nevertheless, it remains an intriguing area for future research to analyze the dynamics and potential equilibria that arise when all data consumers adopt the same strategy.

Thank you for your encouraging and insightful feedback. Below, we provide point-by-point explanations to key questions raised in the comments.

W1): The system uses the SPSB auction mechanism, why is not bidding the value truthfully optimal? That is the main promise of this type of auction.

In the generalised second price auctions, auction theory [1] proved that bidding the true value of the data resources is optimal towards maximizing the accumulated utility. However, the authors in [2] demonstrated that truthful bidding might not be optimal under budget constraints and limited availability of data resources. In this sense, it is necessary to design bidding strategies for data consumers to automatically calculate bid prices for the available data resources in view of the budget limit, instead of bidding with the prices in the simplified linear form.

W2）How to make sure the utility estimation is accurate?

Accurate utility estimation is crucial for the proposed. In practice, it is recommended to continuously monitor and update the utility estimation model as new bidding data becomes available, ensuring its adaptation to potential changes in the AFL environment.

W3) The experiments could be enhanced by including some auction-based datasets or data from AFL market.

Thanks a lot for your suggestion. Since publicly available data from the AFL market is not accessible, and existing auction-based datasets, such as those from online advertising, are not directly applicable to a federated learning setting, we have followed established methods [3,4] to collect and generate the datasets used in our experiments. We will also explore additional auction-based datasets that could potentially be adapted for a federated learning setting.

W4) How practical is the proposed framework for real-world use?

Indeed, the proposed method can be adapted for use in most FL settings that include an incentive mechanism with data consumers competing to bid for data owners [5,6,7], particularly in auction-based FL scenarios. A practical real-world application is in gas usage estimation within the power generation and delivery industry [8].

1. In page 7 (Figure 3) "MultiBOS-AFL" -> "MBOS-AFL"; the accuracy plots could benefit from clearer legends
2. In page 8 (Table 2), missing the percentage sign

Thanks for your correction and suggestions. We have revised the typos in the manuscript and revised the legend of the accuracy plots to make it clear following your suggestions.

---

[1] Vijay Krishna. Auction theory. Academic press, 2009.

[2] Weinan Zhang et al. Optimal real-time bidding for display advertising. In KDD, 2014.

[3] Yutao Jiao et al. Toward an automated auction framework for wireless federated learning services market. TMC, 2020.

[4] Rongfei Zeng et al. Fmore: An incentive scheme of multi-dimensional auction for federated learning in MEC. In ICDCS, pages 278288, 2020.

[5] Yutao Jiao et al. Toward an automated auction framework for wireless federated learning services market. TMC, 2020.

[6] Rongfei Zeng et al. Fmore: An incentive scheme of multi-dimensional auction for federated learning in MEC. In ICDCS, pages 278288, 2020.

[7] Palash Roy et al. Distributed task allocation in mobile device cloud exploiting federated learning and subjective logic. Journal of Systems Architecture, 113(2):doi:10.1016/j.sysarc.2020.101972, 2021

[8] Hao Sun et al, "HiFi-Gas: Hierarchical Federated Learning Incentive Mechanism Enhanced Gas Usage Estimation," in IAAI-24, 2024.

We sincerely appreciate your thoughtful suggestions and valuable feedback.

1. The selected baselines are outdated to some extent. It is suggested to incorporate more recent ones like [1, 2] with the same setting to help position the proposed method. In addition, Can you compare your bidding strategy in auctions with that of [3], though it is not designed for FL settings, it seems quite relevant. Also, can you comment on peer-based mechanisms (e.g., [4],[5],[6] or the references cited therein) vs auction-based mechanisms?

Both [1] and [2] propose bidding strategies for DCs in the AFL marketplace, but with different objectives. While [1] focuses on revenue maximization, differing from the utility maximization in this paper, the formulations are distinct. [2], however, aims to maximize utility and maintain the AFL marketplace's health, using a reinforcement learning approach similar to the RLB baseline method in this paper.

Both [3] and this paper focus on designing bidding strategies. However, different from [3] which aims to reach an equilibrium between buyers and sellers by designing bidding strategies for both buyers and sellers, we focus on guiding DCs (buyers) to bid for DOs (sellers) by budget management strategies to maximize their KPIs within a given budget. This difference in overall design goals makes it unsuitable to compare [3] (similarly [4][5][6]) with the proposed approach.

2. Limited exploration of communication costs in FL training rounds.

The proposed MSBO-FL framework consists of two main parts: the multi-session budget optimization mechanism (i.e., DO recruitment process) and the FL model training (described in “Preliminaries”).

In the DO recruitment part, computational complexity primarily arises from the utility estimation function and hierachical reinforcement learning (HRL)-based budget allocation phase. The HRL-based budget allocation is with a time complexity of $O(|S|·d + |C_s|·d + m)$, where $|S|$ represents the state space size, $|C_s|$ denotes the number of available qualified DOs in session $s$, $d$ is the neural network dimension, and $m$ is the minibatch size used in the training procedure.

For the utility estimation, the computational demand largely stems from the Shapley value-based contribution evaluation. We employ the Beta Reputation System combined with Shapley Value technique, which has a computational complexity of $O(T·N·log N)$, where $T$ and $N$ are the number of training rounds and data owners, respectively. Efficient contribution evaluation remains a challenging issue in federated learning, where high time complexity is common in related problems.

In the FL model training part, we adopt the FedAvg method, which is widely used in FL research. To compare the computational complexity of different bidding methods for data consumers in AFL, we can focus on the DO recruitment part, as the FL model training generally relies on FedAvg across methods. This recruitment part typically involves utility estimation and other functions, with the distinguishing feature of our proposed MBOS-AFL method being the two-level HRL-based budget allocation that optimizes both inter-session budget pacing and intra-session bidding strategies.

Other Comments Or Suggestions

Thanks a lot for your correction. We have revised the manuscript following your suggestions.

Q1: How does MBOS-AFL perform in scenarios with competing DCs using similar strategies?

If all DCs adopt the same bidding strategy, the differences in their individual historical data could lead to variations in their utility estimation and their states. This variance might result in differences in their bidding behaviors (i.e., actions). Moreover, in practice, different DCs target diverse data owners due to their distinct training tasks. These factors influence their bids through specific parameters and model approaches in their bidding strategies. Nevertheless, it remains an intriguing area for future research to analyze the dynamics and potential equilibria that arise when all data consumers adopt the same strategy.

Q2: Could the method be extended to handle non-static budget constraints (e.g., dynamically changing total budgets)?

MBOS-AFL framework be naturally extended to accommodate dynamically changing total budgets with minimal modifications. The current design already incorporates state representations that track remaining budgets and historical allocations, making it well-positioned to handle budget fluctuations.

Q3: How does MBOS-AFL handle abrupt changes in DO availability across sessions?

The changes in DO availability have limited impact on the operation of MBOS-AFL. This is because such changes directly influence the state observed by the bidding agents, which then adjust their actions based on the learned policy. As a result, MBOS-AFL can dynamically generate appropriate bids in response to variations in DO availability, maintaining robust and adaptive performance under changing conditions.