Distributed Event-Based Learning via ADMM

We appreciate the reviewer for their thorough review and constructive feedback. Below, we address the specific points raised.

• Justification of Experiment Design and Generalization:

The point of our manuscript is to present a distributed optimization/learning algorithm that is both communication-efficient and robust, even under heterogeneous data distributions (e.g., each agent having one digit in MNIST, shown in Fig. 8). The experiments were designed to demonstrate the feasibility of our method in these scenarios. Additionally, we show that our approach scales to larger networks, with examples using 100 clients to train a CNN on CIFAR-10 (Fig. 3) and 50 agents that communicate over a graph (Fig. 12). Our theoretical result (Corollary 2.2) guarantees that the method can indeed scale to large networks. Numerical experiments align with these theoretical predictions, highlighting that our approach is effective even in the presence of 100 agents or more.

• Missing References:

Thank you for bringing these papers to our attention. We will incorporate them into the related work section of the revised manuscript.

The paper by Swaroop et al. (2025) is very recent, so it was not included in our initial submission. Regarding Chen et al. (2022), which discusses the ordered ADMM variant, we cite other relevant works related to our ADMM variant. We will include Chen et al. (2022) in the revised manuscript.

We will also add references to alternative convergence techniques, such as Liang et al., 2022 and Deng et al., 2024. Additionally, (Glasgow, 2022) proves FedAvg is suboptimal under heterogeneity, which aligns with the findings of (Li, 2020c) that we already cite, but we will include this reference as well.

• Communication vs Performance Trade-off:

Our framework allows explicit trade-offs between communication load and solution accuracy, which we demonstrate through experiments in Fig. 8, 9, 11, 12 (App. G). These curves clearly show the relationship between the communication threshold and the resulting model accuracy.

• Benefits of Event-triggered Communication:

Our framework, based on ADMM with event-triggered communication, not only saves communication but also ensures convergence under heterogeneous data distributions. This is not guaranteed by many other federated learning methods. We also provide a bounded error guarantee, which can be controlled by a single threshold value ($\Delta$). Our numerical examples (see, e.g., Fig. 3, as well as 8, 9, 11, 12) highlight that the event-triggered communication strategy requires much less communication for a given target accuracy than a vanilla communication strategy that randomly communicates among neighbors (in the context of our ADMM-based approach).

Clarifications

• Interpretation of $d_{k+1}^i$:

The term $d_{k+1}^i$, defined in line 189, encapsulates the combined effect of the local primal and dual updates. Its change essentially quantifies the deviation of the local state from the last communicated state and serves as the key signal for triggering communication (see (2)).

• Practical Tuning Strategy for $\Delta_k$:

Thank you for raising this point. We discuss a tuning strategy in line 347, where we suggest a time-varying schedule, such as $\Delta_k = \Delta_0 / (k+1)^t$ with $t > 0$. Convergence is further analyzed in App.F. Alternatively, the communication threshold can be actively tuned based on feedback from the system, as in [1].

[1] Cummins, M., Er, G. D., Muehlebach, M. “Controlling Participation in Federated Learning with Feedback” (arXiv:2411.19242).

• Generalization to Decentralized Settings:

We demonstrate in App. G (see Fig. 11 and Fig. 12) that our algorithm operates over a network of agents, supporting decentralized FL systems.

• Covering Asynchrony, Network Delays and Heterogeneous Computing Capabilities:

We already discuss the adaptability of our approach to asynchronous systems at the end of the manuscript (line 412). While initially framed as a synchronous method, event-based communication naturally extends to asynchronous settings, enabling robust operation in real-world scenarios with varying network reliability and synchronization.

We also propose to model the effect of stragglers, network delays and heterogeneous computing capabilities as communication drops, which are effectively managed via our reset strategy. Proposition 2.1, Corollary 2.2 and Theorem 4.1 provide theoretical guarantees for convergence under these conditions.

We believe we have addressed all the questions and provided the necessary clarifications. We kindly request raising our paper’s score based on the responses provided. If there are further questions or additional clarifications needed, please let us know.

We appreciate the time and effort invested in evaluating our work. Below, we address each of the points raised.

• Robustness to Biased/Shifting Distributions:

Our experiments already use heterogeneous datasets with inherent bias and distribution shifts among agents (e.g., each agent having one digit in MNIST, shown in Fig. 8 and other cases in Fig. 3,9,11 and 12).

• Relevance to Safe Zone Design:

We acknowledge the relevance of reducing communication overhead through local conditions in distributed systems. We will incorporate the corresponding reference and the following discussion in the revised version of our manuscript:

Our approach similarly monitors local models through local constraints (our communication event definition, see (2) in the paper), which collectively guarantee the global condition of overall error remaining bounded. While thresholding can be seen as a subset of such local constraints, our methodology is directly inspired by the send-on-delta (SoD) concept (Miskowicz, 2006) and event-based control literature, where communication is triggered by significant state changes.

• Bad Actors and Gradient Poisoning Attacks:

Our method is compatible with existing techniques that mitigate issues related to bad actors and gradient poisoning. Addressing all possible challenges in federated learning is beyond the scope of a single approach. However, our event-based methodology can be integrated with robust aggregation or anomaly detection methods [1,2] to improve security without compromising communication efficiency. We will add these methods into our discussion as possible compatible solutions.

[1] Yin, D., Chen, Y., Kannan, R. & Bartlett, P. (2018). Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates. Proceedings of the 35th International Conference on Machine Learning, 80:5650-5659

[2] Pillutla K., Kakade S. M. and Harchaoui Z. (2022) "Robust Aggregation for Federated Learning," IEEE Transactions on Signal Processing, 70:1142-1154

We thank the reviewer for bringing this up.

• Code in the Submission:

All details related to our implementation can be found in App G. We will release our code if the manuscript gets accepted. However, we are happy to share the current version of the code upon request.

We believe we have addressed all points and agree to add further discussions to the final version. We kindly request raising our paper’s score based on our responses. If further clarifications are needed, we are happy to provide them and refine our work.

We thank the reviewer for their constructive feedback and recognition of our work’s strengths. We reviewed (Hadjicostis et al. 2016), and their approach using running sums offers an interesting alternative to periodic synchronization. While our method ensures strong theoretical guarantees, exploring tools from the gossip literature could be a valuable direction for future work to further mitigate communication drops. We appreciate the reviewer’s insights and are happy to provide further clarifications if needed.