When Large Models Meet Generalized Linear Models: Hierarchy Statistical Network for Secure Federated Learning

However, the related work and background knowledge are not clearly separated from the work's contributions, which include the decentralized fine-tuning framework and the proposed defense mechanism.

1. The description of targeted attacks in Lines 246-247 of the paper is inaccurate. The purpose of the targeted attacks is not to reduce the model’s accuracy for the targets.
2. The paper does not explain the specific setting of the attack methods. Thus, it is difficult to ensure that the attacks themself are effective.
3. This paper should show the experimental results of various attacks without deploying any defense. This can better demonstrate the effectiveness of the proposed approach.
4. This paper needs to conduct more experiments with advanced FL backdoor attack and defense methods, such as and Neurotoxin [1], A3FL [2], FLAME [3].

Reference [1] Zhang, Zhengming, et al. Neurotoxin: Durable backdoors in federated learning. ICML’2022. [2] Zhang, Hangfan, et al. A3FL: adversarially adaptive backdoor attacks to federated learning. NeurIPS’ 2023. [3] Nguyen et al. FLAME: Taming backdoors in federated learning.USENIX Security’ 2022.

1. I feel confused about the application of GLM theory. As described in Section 3, the HStat-Net integrates large pre-trained models with GLMs. However, from the description in Section 5.1, the HStat-Net simply appends two fully-connected layers to the CLIP/ResNet image encoder. Could the authors further explain the relation between the HStat-Net and the Generalized Linear Model? How does the GLM theory be applied here?
2. As described in Section 3, the statistical net performs dimensionality reduction to enhance privacy. Could the authors provide further clarification on how privacy is specifically improved through this process?
3. The experiments mainly utilize CLIP (ViT-B-32) and ResNet-152 as image encoders. Since the paper researches on large pre-trained models, I suggest the authors conduct supplementary experiments on larger CLIP image encoders in OpenCLIP [1] to thoroughly evaluate the performance and generalization capabilities of the approach across different model scales and embedding dimensions.
4. Table 4 seems to be missing a control group without any defense, e.g., FedAvg.

[1] https://github.com/mlfoundations/open_clip

The proposed method is only evaluated on image classification tasks, and the tasks are relatively easy with a small number of classes and total number of samples. In general, it is a bit unclear why HStat-Net is much better than robust aggregation, e.g, https://arxiv.org/abs/1912.1344

1. HStat-Net is designed primarily for classification tasks, which may limit its generalizability to other types of tasks, such as text generation or multi-modal learning.
2. While the paper claims HStat-Net is scalable, the additional computational overhead of integrating GLMs and handling large-scale datasets could pose challenges in real-world applications with resource-constrained devices.
3. This work has provided analysis for total misclassification rate, but it lacks theoretical justification for the convergence guarantee of the proposed method without/with attack.

First thing first, I admit that I am no expert in anomaly detection in federated learning. I would try to comprehend the paper as a general audience. However, I have a very hard time trying to understand the motivation of this paper and why it is special. Here are the reasons:

Weak motivation and novelty

Though this paper's motivation is repeatedly emphasized as based on GLMs, I see little to no relation between this method and GLMs. The following questions could help motivate the paper much better:

1. What is the theoretical connection between the proposed methods and GLMs?
2. Why would the proposed method be specifically motivated by GLMs? Why don't I design methods motivated by other statistical models for anomaly detection, but GLMs?

As far as I understand, the proposed method just adds additional feature extractors to the pre-trained models. The idea here is very common and well-known in the literature. Moreover, the following questions would also help:

1. Why would we want to add additional feature extractors on top of pre-trained models? Why don't I just keep the last 2 or 3 layers of the pre-trained models unfreeze instead?
2. Which properties make the proposed methods special compared to similar ideas on refining the features of pre-trained models in the literature?

Weak experimental setups

Since the authors do not mention other anomaly detection literature in the Related works, the authors do it here. It is not just simply mentioning the name/sources of those methods, but also elaborates on the key features of previous methods, and comments on the key differences between the proposed method and the other methods in the literature. Besides, comments on backdoor attack methods used for experiments, and the intuition why this method works well against those attack methods are also helpful.

In Table 4, please highlight the methods that achieve the best performances (make it bold or red ...). This would make the readers keep track of the outstanding methods more easily.

Superficial Related Works

Section 6 - Related works - is poorly written. The authors should consider adding paragraphs (at least) for the literature on: (1) Federated Learning, (2) Anomaly Detection in Federated Learning, and (3) Large pre-trained models. Besides, there are lots of acronyms that are not defined: GAMs, and LIME.

The writing in that section right now is very uncomprehensive and desultory.

Other errors/typos

The typos are not carefully checked. Here are some examples:

1. The citations in this paper are quite a mess. Please you \citep{(paper)}, \citet{(paper)} instead of just \cite{(paper)}.
2. In Equation 2, should it be $g(\mathbb{E}(Y \mid {\color{red} \mathbf{R}})) = \mathbf{R}^\top \mathbf{w}_{h}$?
3. In Equation 3, should it be $\hat{y}_i = \mathbf{h}_i(\mathbf{s}_i({\color{red} \phi(\mathbf{x})})) = \psi_i(\phi(\mathbf{x}))$?
4. Multiple space typos, for example on page 3, line 109, "...generalizationCai et al."

Conclusion

The above are just uncomprehensive suggestions for this paper. I believe the paper's presentation and soundness could be improved. But as of this state, I cannot recommend an acceptance for this paper.