ZGS-Based Event-Driven Algorithms for Bayesian Optimization in Fully Distributed Multi-Agent Systems

(1). Bayesian optimization (BO) aims to optimize the function indeed, and we design the novel distirbuted BO (DBO) paradigm in order to optimize the global objective function (7) of DBO (equally $f$ in (4)) over a fully distributed multi-agent system (MAS). The whole procedure of BO includes surrogate modeing and derserving next potential profitable locations by acquisition functions. Gaussian process (GP) is the mainstream choice for surrogate modeling. Acquisition functions are utilized to search next potential profitable locations. In our work, Random Fourier features (RFF) are used to replace a typical GP of BO. The reference paper "Dai et al., 2020" illustrates that "A GP with RFF approximation can be interpreted as a Bayesian linear regression model with $\phi(x)$ as the features". However, unlike that W are sampled according to (2) of "Dai et al., 2020" in the FBO setting, $W$ is derived by training in our DBO setting, specifically, training by our presented ZGS-ED-DLCB according to (12). Then the local RFF-based DLCB is utilized to deserve the next profitable location by using the trained $\hat{W}_{t}^{i}(K)$ for each agent in each iteration step.

(2). Thanks a lot for reading the reference paper "Dai et al. (2020)", which is [1] in your mark. The reference paper [2] you mark is "Dai et al. (2021)" in our reference list. Although both tree-structured federated learning (FL) and decentralized (fully) distributed learning (DL) are distinct distributed machine learning paradigms, FBO in FL and DBO in DL are distinct paradigms. FBO/FL algorithms cannot be implemented in the DBO/DL setting. We reference Dai et al. (2021; 2020), however, it's not feasible to conduct comparative experiments using the algorithms presented in those references. We introduce and investigate the novel DBO paradigm, and we present the first DBO algorithm, ZGS-ED-DLCB. No prior DBO algorithms have been developed before. Similar to the comparative experiments in Dai et al. (2020), well-known DL algorithms including the DCL, ATC-LMS, CTA-LMS and ADMM-based algorithms are adjusted to adapt to the DBO setting as the baselines in the comparative experiments.

Moreover, in the FBO experiments, none of the baselines compared in the experiments are federated/distributed BO. The following is a direct quote from the NeurIPS reference paper Dai et al. (2020): "Although FTS is the first algorithm for the FBO setting, some algorithms for meta-learning in BO, such as ranking-weighted GP ensemble (RGPE) [16] and transfer acquisition function (TAF) [55], can be adapted to the FBO setting through a heuristic combination with RFF approximation.". Drawing inspiration from the experimental design in Dai et al. (2020), we have constructed comparative experiments for the first DBO algorithm.

(3). We consider the expensive-to-evaluate black-box optimization problem within a fully distributed multi-agent system (MAS) and create a distributed BO (DBO) algorithm, ZGS-ED-DLCB. Moreover, no prior works inverstigate the problem (including leveraging BO) in a fully distributed MAS. In very recent years, novel federated Bayesian optimization (FBO) algorithms, which precede the development of DBO algorithms, (Dai et al. (2021; 2020)) are developed in the federated learning (FL) setting. Tree-structured federated learning (FL) and decentralized (fully) distributed learning (DL) are distinct distributed machine learning paradigms. Similiar to that Dai et al. (2020) presents the FBO paradigm and the first FBO algorithm in the FL setting, we propose the DBO paradigm and the first DBO algorithm in the DL setting. In contrast to the FBO in the FL setting, DBO faces unique challenges to tackle, which are illustrated in the Introduction section. At the end of Introduction section, the main contributions are summerized.

(4). In response to your concerns, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

Thanks for your interests and comments.

Response:

1. ${W_t^i}^*$ is derived by training and used in each iteration step of ZGS-ED-DLCB. Specifically, it is utilized in the calculation of local mean function $m_{i, t}\left(\mathrm{x}\right)=S_t^i {W_t^i}^*$ (please refer to the equation (5) in our work) for each agent. The calculation of $m_{i, t}\left(\mathrm{x}\right)$ is eaisier and more resource-saving than that of the FBO algorithms. In terms of the aforementioned corresponding or relevant content, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

2. The problem formlation can be the same for FBO and DBO. However, while FBO uses a tree-structured paradigm with a cloud server, DBO is a decentralized paradigm without the use of a server. The FBO setting includes a cloud server, while no server is leveraged in the DBO setting. The procedure of an FBO algorithm includes the agent side and the server side, whereas the procedure of a DBO algorithm only involves the agent side. The distinction extends beyond just the network topology. There are specific communication mechanisms and limitations in the settings of fully distributed learning (DL) and federated learning (FL), not just in the aspect of network topology. Therefore, FBO algorithms are not incompatible with the DBO setting, and vice versa. FL origins much later than DL, although they are both distributed machine learning paradigms. According to your concern, why are the FBO algorithms not compared with the adapted DL algorithms in the FBO works? It seems that comparative experiments involving the adapted well-known DL algorithms (should be adapted as you expect) in the FL setting should be implemented. But the authors of the FBO works do not handle these and it is not unreasonable due to the distinction of the DBO and FBO settings.

3. Thank you for noticing the phrase "the first". We highlight the sentence in our abstract "Moreover, we propose a novel generalized fully distributed convergence theorem, which represents a substantial theoretical and practical breakthrough wrt the ZGS-based DL." to emphasize that our ZGS-ED-DLCB algorithm overcomes the drawback exists in the recent ZGS-based DL algorithms that we notice theoretically and practically, in which all agents require the $S_{t}^{i}$ from the other agents to work. ZGS-ED-DLCB is a feasible algorithm in practice.

Response for Strengths:

We appreciate your positive remarks about the problem setting and theoretical analysis of the presented ZGS-ED-DLCB algorithm in our work.

Response for Weaknesses/Questions:

(1) We appreciate your feedback regarding our writing and presentation. We have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

(2) We have revisited our manuscript with a fresh perspective and revised it. DBO is a novel paradigm that we introduce and investigate, no prior DBO algorithms or exist. Could you please display the additional reference papers you think that we should reference to us?

(3) In our paper, the data samples used in the whole experiments are limited to each agent. Compared with the traditional centralized BO, the decentralized learning framework in the DBO setting has the following advantages in terms of privacy: (a) Data Localization: In fully distributed learning, each node processes its own data locally and shares only model updates (like weight updates or gradients), not the raw data. This means that raw data doesn't need to propagate through the network, reducing the risk of data leaks. (b) Reduced Centralization Risk: In centralized learning, all data is brought to a central node for processing. If this central node is compromised, all data could potentially be exposed. In fully distributed learning, there's no such central node, reducing the overall attack surface of the system. (c) Regulatory Compliance: In some cases, the storage and processing of data may be subject to strict regulatory requirements, such as GDPR. In these cases, distributed learning allows for data to be processed locally, making it easier to meet these regulatory requirements.

Response for Minor Comments:

(1) The latest transferred knowledge of agent $i$ is denoted as $\hat{{W_t^i}}(k)=W_{t}^{i}(k_{t,i}^{m_{i}})$. We have revisited our manuscript with a fresh perspective and revised it.

(2) We are the first to present nontions of regret in the DBO setting. In the synthetic experiments, the evaluation metric $R_T/T$, which is associated with the global cumulative regret $R_T$, is used to compare the learning performance of ZGS-ED-DLCB with the baselines is in the paper.

Response for Strengths:

We greatly appreciate your positive remarks on the problem setting, theoretical results and experimental aspects of our work.

Response for Weaknesses/Questions:

In contrast to centralized BO, DBO is the corresponding fully distributed version. In the theory of distributed learning, learning performance of distributed learning equals to that of centralized learning. DBO does not change the regret bound of the centralized BO. Regret bound of the ZGS-ED-DLCB algorithm originates from to the theory of Regret Bound in Section 4.2 of Bogunovic et al. (2016). In terms of the squared exponential (SE) kernel, the regret bound of the ZGS-ED-DLCB algorithm is $R_T=\tilde{\mathcal{O}}(\max\lbrace\sqrt{T}, T\epsilon^{1/6}\rbrace)$ when employing the local LCB acquisition functions and the RFF-based surrogate model according to Corollary 4.1 of Bogunovic et al. (2016). Please see App. M.

Response for Strengths:

We are grateful for your acknowledgement and positive assessment of the theoretical guarantees and the experimental results in our research.

Response for Weaknesses/Questions:

(1). We appreciate your feedback regarding our writing and presentation. Bayesian optimization (BO) aims to optimize the objective function indeed, and we design the novel distirbuted BO (DBO) paradigm in order to optimize the global objective function (7) of DBO (equally $f$ in (4)) over a fully distributed multi-agent system (MAS). The whole procedure of BO includes surrogate modeing and derserving next potential profitable locations by acquisition functions. Gaussian process (GP) is the mainstream choice for surrogate modeling. Acquisition functions are utilized to search next potential profitable locations. In our work, Random Fourier features (RFF) are used to replace a typical GP of BO. The reference paper "Dai et al., 2020" illustrates that "A GP with RFF approximation can be interpreted as a Bayesian linear regression model with $\phi(x)$ as the features". However, unlike that $W$ are sampled according to (2) of "Dai et al., 2020" in the FBO setting, $W$ is derived by training in our DBO setting, specifically, training by our presented ZGS-ED-DLCB according to (12). We call our presented paradigm DBO because we are the first to extend BO to the distributed learning (DL) setting of a fully distributed MAS, similar to that FBO origins by extending BO to the federated learning (FL) setting (please see the rederence paper "Dai et al., 2020", for example, the Abstract section).

Naturally and necessarily, $W$ is time-varying in the DBO setting. $t$ represents the iteration index of ZGS-ED-DLCB and local BO. After one iteration, one potential location and its corresponding output is derived by each agent. Then, the corresponding $S_t^i$ is changed because the available data samples are increased to update $S_t^i$. Then $W$ are also necessarily changed by the next iteration step of ZGS-ED-DLCB. Thus, $W$ should be time-varying.

We present the novel DBO paradigm, and we present the first DBO algorithm, ZGS-ED-DLCB. No prior DBO algorithms have been developed before. Both DBO and FBO are paradigms in the field of distributed machine learning. FL is tree-structured and DL is decentralized, so FBO algorithms cannot be implemented in the DBO setting. Similar to the reference paper "Dai et al., 2020", well-known DL algorithms including the DCL, ATC-LMS, CTA-LMS and ADMM-based algorithms are adjusted to adapt to the DBO setting as the baselines in the comparative experiments. Moreover, in the FBO experiments, none of the baselines compared in the experiments are federated/distributed BO.

(2). In contrast to centralized BO, DBO is the corresponding fully distributed version. In the theory of distributed learning, learning performance of distributed learning equals to that of centralized learning. DBO does not change the regret bound of the centralized BO. Regret bound of the ZGS-ED-DLCB algorithm originates from to the theory of Regret Bound in Section 4.2 of Bogunovic et al. (2016). In terms of the squared exponential (SE) kernel, the regret bound of the ZGS-ED-DLCB algorithm is $R_T=\tilde{\mathcal{O}}(\max\lbrace\sqrt{T}, T\epsilon^{1/6}\rbrace)$ when employing the local LCB acquisition functions and the RFF-based surrogate model according to Corollary 4.1 of Bogunovic et al. (2016). Please see App. M.

(3). $\hat{{W_t^i}}(k)$ is the local transferred weight vector at the $k$th sub-iteration step during the $t$th iteration step. To illustrate our ZGS-ED-DLCB algorithm more detailedly, the pseudocode is shown in App. C. From the pseudocode, we can see the the update of $W_{t}^{i}(k)$ includes $T$ iterations from 1 to $t$, of which each includes $K$ sub-iterations from 0 to $K$ in Stage 1. Thus, $t$ indexes the iteration, $k$ indexes the sub-iteration. The parameters are time-varying because lcoal data samples are actually increased after each iteration step indexed by $t$ and the parameters corresponding to the calculation process of ZGS-ED-DLCB in each iteration step are changed with the index $t$.

(4). In response to your concerns, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.

(5). RFF are utilized to approximate a typical, so the approximated GP posterior mean and variance are calculated in this paper. Similar to the work Dai et al., (2020), the posterior mean and variance are calculated according to formulae (6) and (7) in App .B of Dai et al., 2020. The corresponding computing formula for our ZGS-ED-DLCB in the DBO setting will be shown in our revised manuscript.

(6). In response to your concerns, we have revisited our manuscript with a fresh perspective and revised it. If you are interested, the final version will deliever the revised manuscript.