LayerGuard: Poisoning-Resilient Federated Learning via Layer-Wise Similarity Analysis

Thank you very much for the valuable comments. Please find our responses below.

Q1＆Q2: The paper fails to clearly articulate how LayerGuard uniquely addresses the problem beyond previous layer-wise strategies. The motivation remains vague and feels incremental, especially considering that previous work has already hinted at per-layer granularity as a promising direction.

＆

While the layer-wise cosine similarity metric (LCSS) and the joint weighting mechanism are well-formulated, the key ideas seem like relatively straightforward extensions of prior defenses (e.g., MultiKrum, FLDetector). The paper lacks a strong theoretical insight or technical innovation that would mark a significant advancement in the field.

R1＆R2: Thank you for the insightful comment. TLP attacks are not applied to the entire model but rather to specific layers. In contrast, most existing defenses perform analysis at the whole-model level. Therefore, to defend against TLP attacks, one naturally considers fine-graining the defense by shifting the perspective from the entire model to individual layers. This idea is intuitive and easy to come up with, as it follows a natural progression. It represents the first and most straightforward step toward addressing TLP.

However, simply recognizing this step is far from sufficient to build a defense that is stable, robust, and effective against TLP attacks. Every step beyond this point introduces substantial challenges and presents meaningful research problems：

1、Different layers have functional, parametric, and update-scale differences. As a result, malicious updates may behave differently across layers. Even assuming that a method like Layer-wise MultiKrum already knows which layers are poisoned, it still struggles to identify malicious updates in some layers—because in certain layers, the Krum Distance of benign and malicious updates are almost indistinguishable. This issue is detailed in Appendix-14 of [11]. Therefore, designing a robust metric to amplify the difference between benign and malicious updates is a highly challenging problem. To address this, LayerGuard proposes LCSS, which is not a simple cosine similarity, but a local similarity–based averaging cluster metric designed to mine the collusive behavior of attacks. LCSS can detect malicious updates in a way that is independent of the specific properties of different layers. Experiments also validate its superior performance.

2、Analyzing only intra-layer information is insufficient to defend against TLP attacks, as confirmed by our ablation study in Section 4.4. Layer-wise MultiKrum only analyzes information within each layer. In contrast, our method not only analyzes intra-layer information but also innovatively proposes to combine inter-layer information. In the aggregation stage, LayerGuard introduces a joint weighting mechanism to assess the credibility of each client in each layer from two complementary perspectives:User-level weights capture inter-layer behavior,Layer-specific weights reflect intra-layer behavior.

3、The most insidious aspect of TLP attacks is that they affect only selected layers while leaving others largely benign. Suppressing malicious updates in targeted layers without disrupting benign ones in non-targeted layers is highly challenging. Unlike Layer-wise MultiKrum, which applies a fixed treatment strategy across different layers, LayerGuard performs adaptive handling. In the Layer Update Boundary Definition phase, each layer is assigned its own benign boundary, allowing reasonable weight assignment even for benign updates coming from malicious users (i.e., for non-targeted layers).

The solutions we propose to address the above research challenges are the true innovations of our approach. The key idea of LayerGuard is not a straightforward extension of prior methods like MultiKrum or FLDetector, but a defense framework designed to tackle TLP-specific difficulties in a principled and robust way.

Q3: The experimental evaluation only includes baselines published between 2017 and 2022 (Krum, Trimmed Mean, FLAME, FLTrust, etc.). No baseline from 2023 - 2025 is considered.

R3: Thank you for this valuable suggestion. In the main experiments, we have added two stronger and more recent baselines for comparison:

Xu et al. (2024) – Dual Defense: Enhancing Privacy and Mitigating Poisoning Attacks in Federated Learning, NeurIPS 2024.

Yan et al. (2023) – RECESS Vaccine for Federated Learning: Proactive Defense Against Model Poisoning Attacks, NeurIPS 2023.

Following the suggestion in the original RECESS paper, we adopt the variant RECESS with Median when evaluating against MPAF and backdoor attacks.

Under untargeted attacks(The additions in Table 1):

Under LP attack(The additions in Table 2):

RECESS demonstrates strong defense performance against untargeted attacks, but still falls short compared to LayerGuard. This robustness stems from LayerGuard’s fine-grained evaluation of client credibility across layers. Under the LP attack, both RECESS and DDFed fail to defend effectively. This is because defenses based on whole-network analysis are not sufficient to counter the layer-wise fine-grained nature of TLP attacks.

Thank the authors for their rebuttal. Most of my previous concerns have been adequately addressed, and I will revise my score accordingly (before the official deadline). However, I still have some concerns regarding the theoretical robustness of the proposed method. Specifically, while the addition of a user-level filtering mechanism is intuitively appealing, it remains unclear whether this is sufficient to fundamentally overcome the limitations of layer-wise detection. For example, in scenarios where only a few layers are poisoned and these poisoned updates are crafted to closely resemble benign ones—as the authors themselves note (R1&R2-1)—such attacks may remain indistinguishable under the current framework. I would encourage the authors to provide a more rigorous theoretical analysis to better support the robustness claims of their method in such cases.

We thank the reviewer for the thoughtful comments. We provide our responses below.

Q1: It is recommended that the authors include theoretical analyses of the LCSS or weighted mechanism, providing theoretical bounds on aspects such as convergence or robustness.

R1: We appreciate the reviewer’s suggestion regarding the theoretical analysis of LCSS and the proposed weighting mechanism. While LayerGuard is currently motivated and validated from a practical perspective—demonstrating robust empirical performance under a wide range of attacks and settings—we acknowledge that formal theoretical guarantees (e.g., convergence or robustness bounds) would further strengthen the framework's rigor and completeness.

However, providing such analysis is non-trivial due to the layer-wise structure and adaptive weighting scheme involved, which introduce strong interdependencies across clients and layers. We believe that a comprehensive theoretical analysis requires dedicated future work, potentially combining tools from robust statistics and federated optimization theory.

We have clarified this limitation and our plan to explore it further in future versions of the paper.

Q2: The paper mentions a potential slight decrease in LayerGuard's accuracy when no attacks are present. This observation requires a clearer explanation and quantification. Furthermore, an analysis of the trade-off between defensive robustness and benign performance for the system would be beneficial.

R2: We appreciate the reviewer’s suggestion. As briefly explained in Sec. 3.5 and Sec. 4.2, we now provide a more detailed clarification.

In the absence of malicious clients, the Anomalous User Identification module still filters users based on LCSS in each layer. As a result, some benign users with relatively high LCSS values may be mistakenly included in the anomalous user set. These users will be assigned lower weights in both the user-level and layer-specific weighting processes, which leads to a slight drop in accuracy even without attacks.

The High-Risk Layer Detection mechanism helps alleviate this issue. Since all users are benign, it is rare for any client to consistently exhibit high LCSS across multiple layers. Therefore, even if a benign user appears anomalous in a certain layer, it will not be assigned a significantly low user-level weight, which helps maintain overall performance in the benign setting.

Q3: The authors did not clarify the number of classes for the three datasets in Table 1. It is recommended to use datasets with different class numbers to enhance the credibility of the experimental results.

R3: Thank you for pointing this out. We clarified the number of classes for FashionMNIST and CIFAR-10 in Appendix B.1, but we did not explicitly state that CINIC also contains 10 classes. We will make this clearer in the revised version.

As for your suggestion to include datasets with varying numbers of classes to enhance the credibility of the results, we agree that this is a valuable direction. We plan to explore this aspect in future work to further validate the generality of LayerGuard under diverse classification settings.

Q4: The paper currently lacks a detailed quantitative analysis of LayerGuard's computational overhead. We request the authors to provide a comprehensive breakdown of their computational complexity and practical runtime performance to thoroughly assess their scalability in large-scale federated learning deployments.

R4: Thank you for raising this important concern. We now provide a detailed analysis of LayerGuard's computational complexity and its practical scalability in federated learning. The dominant cost of LayerGuard arises from computing pairwise cosine similarities per layer. Let $n$ be the number of clients, $L$ the number of parameter layers (e.g., weights and biases), and $\bar{D}$ the average parameter dimensionality per layer. The overall time complexity of LayerGuard is:

The first term corresponds to computing cosine similarities between every pair of client updates at each layer, and the second term stems from the final weighted aggregation step. The rest of the pipeline (user-level weight computation, layer-specific weight adjustment, etc.) contributes only $O(L \cdot n + n \log n)$, which is negligible compared to the similarity computations.

We evaluated the average per-round runtime of LayerGuard on different models under untargeted attack settings.

We believe that LayerGuard is scalable in large-scale federated learning (FL) systems. Although it theoretically incurs a quadratic cost with respect to the number of clients $n$, i.e.,

the design of LayerGuard is naturally parallelizable. Since the similarity computations and subsequent analysis are independent across layers, both the cosine similarity computations and layer-wise aggregation processes can be distributed across multiple computational threads or devices.

This parallel structure enables efficient execution even in high-dimensional and large-client FL scenarios, making the overhead practically manageable on modern hardware.

Q5: There are a few minor formatting issues throughout the paper. For instance, on page 4, "i, j ∈ 1, . . . , N" is missing parentheses and should be corrected to "i, j ∈ {1, . . . , N}".

R5: Thank you for pointing out the formatting issue. We will revise the notation on page 4 from "i, j ∈ 1, . . . , N" to "i, j ∈ {1, . . . , N}" and carefully check the rest of the paper to fix any remaining minor formatting inconsistencies in the final version.

Q6: The paper in section 4.3 states that LayerGuard requires more than one malicious client to be effective. Could the authors please elaborate on the fundamental reasons for this limitation?

R6: Thank you for the question. This limitation arises from the core idea of LCSS, which is to exploit the observation that the similarity among malicious updates tends to be higher than that among benign updates. When there is only one malicious client, this intra-group similarity pattern no longer exists, making it difficult for our method to distinguish the malicious update from benign ones. Therefore, LayerGuard is only effective when there are at least two malicious clients.

Q7: The benign boundary calculation in LayerGuard relies on the assumption that malicious clients constitute less than 50% of the total. Please clarify the specific impact on LayerGuard's effectiveness if the actual poisoning rate exceeds this threshold.

R7: Thank you for your question. If the proportion of malicious clients exceeds 50%, the computation of the benign boundary in the Layer Update Boundary Definition step becomes unreliable, as it may include malicious updates. This contamination can cause the estimated benign boundary to become abnormally high. As a result, some malicious updates may appear close to or even below the boundary, and thus be assigned relatively high layer-wise weights—potentially even reaching 1. Consequently, the influence of these poisoned updates may not be properly suppressed, leading to a rise in the Backdoor Success Rate (BSR).

Q8: LayerGuard assigns user-level weights based on the number of high-risk layers identified for a client. However, is it possible for a malicious client to target only a small number of layers? In this case, the user-level weight assigned by the scheme would be relatively large, even though it should be smaller in practice.

R8: Thank you for pointing this out. This situation—where a malicious client only poisons a small number of layers—can indeed occur, especially when the model is approaching convergence. However, LayerGuard’s High-Risk Layer Detection mechanism is specifically designed to mitigate such cases. When attacks are concentrated on only a few layers, those layers still tend to stand out in the similarity analysis and are likely to be identified as high-risk.

Even if some detection error occurs, as discussed in Sections 3.5 and 3.6, the number of high-risk layers for such clients typically remains low. This still results in a noticeable penalty in user-level weight, ensuring that their poisoned updates receive lower aggregation weights. Thus, the impact of malicious updates remains effectively suppressed..

Thanks to the reviewers' insightful suggestions, which are helpful to strengthen the completeness of this work.

Q1: Conduct thorough ablation studies for all threshold-related parameters and justify design choices.

R1.1-τ₀ and step size: In our method, the parameters τ₀ = 0.95 and step size = 0.05 are used to detect anomalous users that exhibit high Layer-wise Cosine Similarity Scores (LCSS) in each layer. Since different layers may have different parameter distributions, the LCSS of malicious updates can vary significantly across layers. For example, in layer A, malicious updates may lie in the range [0.90, 0.94], while in layer B they may lie in [0.80, 0.83].

Therefore, starting from τ₀ and gradually decreasing it with a fixed step size allows us to adaptively identify anomalous users in each layer while avoiding misclassification of benign users. In this work, we set τ₀ = 0.95 and step size = 0.05 based on the trends observed in Figure 2, as explained in Section 3.4.

Below, we conduct an ablation study to demonstrate that selecting a reasonable τ₀ and step size is a flexible process, and it becomes even easier with the help of distributions such as those shown in Figure 2. The focus is on step size, since τ₀ = 0.95 is simply the result of the first decrement from 1 (i.e., 1 − 0.05 = 0.95). Therefore, we experimentally evaluate how the step size affects the defense performance.

The experimental settings are the same as those in Section 4.3. From the results, we observe that when the step size is set between 0.05 and 0.1, the defense performance shows no significant difference. A smaller step size leads to a higher BSR, because it may fail to cover most of the malicious users' LCSS, resulting in some malicious users being assigned larger user-level weights, which in turn increases the BSR. On the other hand, a larger step size reduces accuracy, because although it can cover malicious updates, it may also mistakenly include some benign updates. This causes certain benign users to be assigned smaller user-level weights, ultimately degrading the accuracy of the main task.

R1.2-$\gamma_{\text{mal}}^l$: The Layer-Specific Weight Assignment is essentially a process of assigning linearly decreasing penalty weights based on the LCSS values. Any user with an LCSS value higher than $\gamma_{\text{mal}}^l$ will be assigned a layer-specific weight of 0. In Anomalous User Identification, we determine the set of anomalous users in each layer — that is, a group of users with relatively high LCSS values. Therefore, the most reasonable and intuitive choice for $\gamma_{\text{mal}}^l$ is to set it as the minimum LCSS value within the set of anomalous users.

In this way, all users in the anomalous set—whose LCSS values are greater than or equal to the minimum—will be assigned the lowest layer-specific weight, i.e., 0.

We now conduct an ablation study to evaluate how different settings of $\gamma_{\text{mal}}^l$ affect the defense performance. Specifically, we test four strategies for setting $\gamma_{\text{mal}}^l$: the maximum, median, average, and minimum LCSS values within the anomalous user set.

The experimental settings are the same as those in Section 4.4. As the value of $\gamma_{\text{mal}}^l$ increases, malicious updates in the anomalous user set whose LCSS values are lower than $\gamma_{\text{mal}}^l$ will be assigned larger layer-specific weights, which ultimately leads to a higher BSR.

Q2: Clearly define the threat model and discuss the assumptions under which this defense is expected to work.

R2: We sincerely apologize for not clearly stating our threat model earlier. We will add a dedicated Threat Model section after Related Works in the revised version to clarify this. Below, we provide a concise description of the threat model.

For Attacker, Attacker’s Goal. The attacker’s goal is to perform both untargeted model poisoning attacks and targeted layer poisoning (TLP) attacks. It is important to note that we do not define TLP as a backdoor attack in general. In this paper, the introduced LP attack (Section 2.2) is a backdoor-based TLP attack, and we do not discuss other forms of TLP attacks because this fine-grained poisoning strategy has only recently been proposed. The LP attack is the first and, to our knowledge, the only representative of such attacks so far. Nevertheless, we believe that our fine-grained detection method is general and can be extended to future variants of TLP attacks.Attacker’s Capabilities. The attacker controls a subset of clients and can fully manipulate their gradient updates in each round. The attacker has access only to local training information of these clients and cannot compromise or interfere with the server.Attacker’s Knowledge. The attacker has no knowledge of the gradient updates of benign clients.

For Defender, The defender (i.e., the server) does not know the distribution of client-side data, nor the number or identities of malicious clients. The only available information is the uploaded gradient updates from all participating clients.Defender’s Assumptions. There are two key conditions for the defense to be effective: First, the number of attackers must be greater than one, because the core idea of LCSS is to capture that the similarity among malicious updates is higher than that among benign updates. Second, the number of attackers must be smaller than the number of benign users, which is a necessary condition for the High-Risk Layer Detection and Layer Update Boundary Definition to function correctly.

Q3: Evaluate the method against stronger, more recent baselines.

R3: Thank you for this valuable suggestion. In the main experiments, we have added two stronger and more recent baselines for comparison:

Xu et al. (2024) – Dual Defense: Enhancing Privacy and Mitigating Poisoning Attacks in Federated Learning, NeurIPS 2024.

Yan et al. (2023) – RECESS Vaccine for Federated Learning: Proactive Defense Against Model Poisoning Attacks, NeurIPS 2023.

Following the suggestion in the original RECESS paper, we adopt the variant RECESS with Median when evaluating against MPAF and backdoor attacks.

Under untargeted attacks(The additions in Table 1):

Under LP attack(The additions in Table 2):

RECESS demonstrates strong defense performance against untargeted attacks, but still falls short compared to LayerGuard. This robustness stems from LayerGuard’s fine-grained evaluation of client credibility across layers. Under the LP attack, both RECESS and DDFed fail to defend effectively. This is because defenses based on whole-network analysis are not sufficient to counter the layer-wise fine-grained nature of TLP attacks.

Q4: Test the defense under general backdoor attacks, beyond just layer-specific ones, to demonstrate broader applicability.

R4: Our work primarily focuses on Targeted Layer Poisoning (TLP) attacks — a newly proposed and highly stealthy and promising poisoning strategy — as well as untargeted attacks, which typically cause more severe degradation in model performance.

Nevertheless, to demonstrate LayerGuard’s versatility, we have additionally conducted experiments against constrain-and-scale, a representative general backdoor attack.

Q5: Revise and validate the experimental setup, particularly in the benign case, to ensure baseline accuracy is reasonable.

R5: We appreciate the reviewer’s concern. The experimental setup in our main experiments mostly follows the configurations referenced in [26] and [11]. The reproduced baseline results are consistent with those reported in their original papers, which supports that the baseline accuracy is reasonable.

We believe the relatively lower Mean Accuracy under the benign setting is primarily due to the high degree of non-IID data distribution in this setup. This reflects a more realistic and challenging federated learning scenario.

Thanks to the authors for their efforts in addressing my concerns. While I appreciate the idea of incorporating layer-wise and client-wise information, my main concern about the method's heavy reliance on heuristic thresholding still remains. There are multiple heuristic thresholds involved, and the process often follows a "try-until-it-works" approach.

For example,

"We then determine the final anomalous set $B_l$ by selecting the largest threshold $\tau$ in the descending sequence $\{\tau_0, \tau_0 - 0.05, \ldots, 0\}$ such that $f_l(\tau)$ is non-empty."

• It’s unclear why the first threshold that yields a non-empty set is sufficient to capture all malicious clients, especially in the absence of theoretical justification.

• Similarly, for Eqn. (5)—is it possible that the resulting set contains only one layer? If so, how reliable is the method when only one layer is flagged as highly risky?

• Another point: based on Figure 2 and the overall methodology, malicious clients tend to exhibit the highest LCSS values on the attacked layers. Doesn’t this imply that their model updates are also quite similar? If so, is there an implicit assumption that malicious clients are aligned in their updates? Would the method still work if the attackers are not coordinated, or use adaptive strategies to intentionally reduce similarity?

• Regarding DDFed, while I recognize its not strong under LP attacks, the results for untargeted attacks are missing. Including them would give a clearer picture of its general robustness.

• Finally, although the added results with the constrain-and-scale attack are appreciated, they alone aren’t enough to demonstrate the method’s generalizability to broader backdoor attacks. Many modern backdoor attacks avoid obvious scaling and keep poisoned models close to benign ones. Can the method still detect such subtle attacks?

• Overall, I believe the paper still needs to better clarify the applicability of the method, and explain why the current set of heuristic thresholds is sufficient to consistently detect malicious clients across various attack types.

I would appreciate if the authors could further discuss these points.

We thank the reviewer for the thoughtful comments. We provide our responses below.

Q1: LayerGuard computes pair-wise cosine similarities for every layer, performs clustering, and maintains per-layer weighting. The paper gives no timing, memory-footprint, or communication-round analysis, leaving open whether it scales to large models or hundreds of clients.

R1 Thank you for raising this important concern. We now provide a detailed analysis of LayerGuard's computational complexity and its practical scalability in federated learning. The dominant cost of LayerGuard arises from computing pairwise cosine similarities per layer. Let $n$ be the number of clients, $L$ the number of parameter layers (e.g., weights and biases), and $\bar{D}$ the average parameter dimensionality per layer. The overall time complexity of LayerGuard is:

The first term corresponds to computing cosine similarities between every pair of client updates at each layer, and the second term stems from the final weighted aggregation step. The rest of the pipeline (user-level weight computation, layer-specific weight adjustment, etc.) contributes only $O(L \cdot n + n \log n)$, which is negligible compared to the similarity computations. LayerGuard exhibits a time complexity comparable to that of some advanced defenses (e.g., Krum), and the overhead remains acceptable even in the presence of multiple clients.

We evaluated the average per-round runtime of LayerGuard on different models under untargeted attack settings.

We believe that LayerGuard is scalable in large-scale federated learning (FL) systems. Although it theoretically incurs a quadratic cost with respect to the number of clients $n$, i.e.,

the design of LayerGuard is naturally parallelizable. Since the similarity computations and subsequent analysis are independent across layers, both the cosine similarity computations and layer-wise aggregation processes can be distributed across multiple computational threads or devices.

This parallel structure enables efficient execution even in high-dimensional and large-client FL scenarios, making the overhead practically manageable on modern hardware.

Q2: Authors state that their method works when malicious number of clients are larger than 1, which we accept as the limitation. However, what if there are multiple malicious clients and they target different layers of the model each round. I doubt that the proposed method would be able to detect it. This might be considered as an adaptive attack as well.

R2: Thank you for raising this insightful point.

In TLP attacks, the choice of poisoned layers is typically driven by specific objectives. For instance, in Layer Poisoning (LP) attacks, layers are selected based on their influence on the Backdoor Success Rate (BSR)—layers with higher impact on BSR are more likely to be targeted. This leads to a natural convergence in target layer selection across attackers, as discussed in Section 5.4 of [11]. That is, to mount an effective attack, adversaries tend to poison the same or similar subset of sensitive layers.

Under the scenario you proposed—where multiple malicious clients each poison different layers in each round—our method's detection accuracy may decrease due to reduced anomaly aggregation in individual layers. However, this strategy also dilutes the strength of the attack itself, making it harder for the malicious gradients to shift the global model meaningfully. As a result, there exists a trade-off for the attacker: while attempting to evade detection by distributing poisoned layers, the attack’s effectiveness is weakened and more likely to be neutralized during aggregation.

This phenomenon highlights an important strength of LayerGuard: by enforcing fine-grained layer-wise analysis, it encourages adversaries to spread their efforts thinly across layers, which in turn reduces attack efficacy.