Directional Rank Reduction for Backdoor Defense

Weakenesses:

W1: In the first equation on Page 3, it seems feasible to do the defense by reducing the norm of the residual matrix to align the benign and poisoned features seems feasible. The features from benign examples move towards the backdoored features. Does the movement hurt the model's clean performance?

Answer: As we points out in our paper, rank reduction is an extension of neuron pruning (which is also a modification of the weight matrix). Established neuron pruning techniques, such as CLP and EP, have been demonstrated to maintain model performance effectively post-pruning. Our rank reduction, which 1) remove only one rank (instead of multiple ranks in neuron pruning) and 2) remove more targetedly to the direction that related to the backdoor behavior should intuitively affect less to the performance than pruning methods in general. Empirically, we show that our method affects the less on the performance compare to the previous method.

W2: The last equation on Page 4 has a strong assumption that all the clean examples are centered around the mean of them. Namely, the method assumes that the distances from all the clean examples to the example center are the same. The examples marked as yellow in Figure 1 are distributed like a circle. However, the real-world data distribution often deviates from the assumption. The distribution could be elliptical-like. In this case, the obtained v is not optimal anymore.

Answer: We appreciate the opportunity to clarify a potential misunderstanding highlighted by the reviewer regarding the assumptions underlying our method. Contrary to the reviewer's interpretation, our approach does not assume uniform distances of all clean examples from their mean, nor does it presuppose a circular distribution of data points as depicted in Figure 1. The shape of the ellipse in a multivariate Gaussian distribution is determined by its covariance matrix. Only when the covariance matrix is isotropic, which means the variance along all directions are the same, the data is distributed like a circle as mentioned by the reviewer. However, we do not put any constraints on whether the covariance is isotropic or not. The only assumption about the covariance matrix we made is assumption 2, which limits the maximum variance of the data distribution. Hence, an elliptical-like distribution is considered in our method, where the Gaussian distribution has a covariance matrix that is not isotropic.

The equation on page 4, which the reviewer refers to, formulates an optimization problem. The objective of this problem is to determine an optimal unit direction vector that maximizes the third central moment when data is projected onto this vector. The stipulation that the vector be of unit length is a constraint applied to the direction vector itself, rather than to the data samples. This is a standard practice in such optimization problems to ensure the direction vector is normalized and hence, the focus is on the direction rather than the magnitude.

W3: In the third row of Table 2, DRR achieves a better trade-off. Why it demonstrates a higher accuracy (ACC) instead of a lower ASR?

Answer: We appreciate the reviewer's insightful observation and recognize the necessity of providing a clearer explanation in our manuscript. The comparatively minimal impact on model performance observed in our study can be attributed to our method's strategy of removing at most one rank per layer. This approach is considerably more conservative than traditional neuron pruning methods, which often entail the removal of multiple ranks. If the number of ranks is tuned to more according to specific scenario, DRR can achieve even lower ASR.

W4: This approach requires the optimization of a vector in each layer, which could be expensive.

Answer: We appreciate the inquiry regarding the computational efficiency of our approach. Primarily, the optimization process can be parallelized across different network layers, offering a substantial decrease in the required time for optimization. Furthermore, in scenarios with high feature dimensions or large dataset sizes, dimensionality reduction through Principal Component Analysis (PCA) can be employed prior to the directional learning phase. This step effectively reduces the computational burden. Subsequently, the learned direction within this reduced space is projected back onto the original space, thereby economizing on memory and computational demands while preserving the integrity of the optimization process.

Thank the authors for providing a detailed rebuttal. However, I still have concerns regarding the motivation of the method.

Figure 1 illustrates why the proposed method works, which is only valid when the data distribution is circle-like. In real-world datasets, the distribution is not the case. Namely, the motivation or the explanation of the proposed method is not convincing anymore.

I understand that the authors do not make any assumptions about the data explicitly. But please check the motivation illustrated in Figure 1. The illustration does not make sense for real-world data distribution.

Hence, I tend to keep my original score.

We would like to thank the reviewer for their appreciation of our work. Below, we have provided our response to the reviewer's concerns.

Weakenesses:

W1: Although this work is interesting, it has a limitation. This paper assumes the defender can get access to the backdoored image. However, this is hard to get in actual situations and thus limits its use greatly. I wonder whether it works without these backdoored data.

Answer: First, to answer the reviewer's final question, the rank reduction framework can be adpated to any scenarios with or without backdoored data, but the metric we adopt to obtain the direction, i.e., third central moment, requires the access to the backdoored data. If other metrics is later being invented to obtain the direction, then the method could be backdoored data-free.

Second, we want to argue that the scenario in which the defender has access to the full dataset is a prevalent assumption within the backdoor attack research community, exemplified by the concept of adopting a third-party dataset as discussed in [1]. Some of methodologies that employ this setting are outlined in: [2, 3, 4, 5].

[1] Li, Y., Jiang, Y., Li, Z. and Xia, S.T., 2022. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems. [2] Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I. and Srivastava, B., Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. [3] Zheng, R., Tang, R., Li, J. and Liu, L., 2022. Pre-activation Distributions Expose Backdoor Neurons. Advances in Neural Information Processing Systems, 35, pp.18667-18680. [4] Tran, B., Li, J. and Madry, A., 2018. Spectral signatures in backdoor attacks. Advances in neural information processing systems, 31. [5] Hayase, J., Kong, W., Somani, R. and Oh, S., 2021, July. Spectre: Defending against backdoor attacks using robust statistics. In International Conference on Machine Learning (pp. 4129-4139). PMLR.

W2: The backdoor attacks that this paper test is not enough. I suggest the authors to test the newest input-specific backdoor attacks in 2022. It's important to identify whether this method can achieve SOTA.

Answer: Actually, the experiments with IAB and WaNet, which are both input-specific backdoor attacks, are in the paper. Please refer to Table 2 in our submitted paper to see our results tested on input-aware dynamic attack (IAB) and Warping-based backdoor attack (WaNet).

Moreover, we conduct more experiments on other types of attacks (AdaptiveBlend, SIG and Smooth), where the results is as shown below:

We would like to thank the reviewer for the detailed review. We will make changes based on the feedback. Please see our responses:

Weaknesses:

W1: More experiments can be conducted (BadNet, Blended, CLA, WaNet, and IAB are insufficient.) The authors can consider attacks like SIG [1] and low frequency (Smooth) [2]. Since your method also took latent separability as an assumption, Adapt-blend and Adapt-patch attacks [3] should also be considered. Evaluating the robustness of adaptive attacks that try to evade the defense would be useful for understanding limitations.

Answer: We've conducted experiments according to the reviewer's suggestions. The results are shown below:

Note that the latent separability mentioned in the paper [3] only considers the latent space in the penultimate layer, while our methods utilize the separability within each layer of the model. This makes our method effective even when the penultimate layer feature is inseparable.

W2: The references and notations should be clarified. For example, what is the reference to Proposition 1?

Answer: The reviewer's request for clarification on the references and notations is acknowledged. However, regarding the reference for Proposition 1, it should be noted that to the extent of our understanding, Proposition 1 is introduced for the first time in our manuscript. Consequently, there are no prior publications to cite for this proposition.

W3: Also, the readability and organization of this paper need to be improved. It is better if an algorithm is provided.

Answer: We appreciate the feedback regarding the clarity and structural aspects of our manuscript. Recognizing the value that an algorithmic representation would add, we have taken the suggestion into consideration and will include a detailed algorithm in the revised draft to enhance comprehension of our proposed method.

We appreciate the reviewer's thorough review and have taken their comments into consideration. Here are our responses to their concerns:

Weaknesses:

W1: Some of theoretical analysis might be not accurate. The utility function is defined using $||R-\gamma (R)||$ and also $||R||-||\gamma (R)||$. However, these two value is not strict equivalent. It also happens in the definition of $E(R)$.

Answer: We thank the reviewer for pointing this out. Indeed, the equation $||R-\gamma(R)||=||R||-\gamma(R)||$ doesn't hold in the general case. However, it does hold when we use the proposed $L_{1, 1}$ norm, as defined in Definition 1. It makes sense if we specify the norm before introducing this equation. We acknowledge that the organization of this section needs to be corrected to avoid misleading the reader.

W2: It is unclear why the 3rd center moment would show the best performance to measure the difference. In other words, would 2nd order moment or 1st order work as well? Since 3rd order is the main metric selected, the author should explain the choice in detail.

Answer: We clarified our choice in the revision of the paper. The first central moment (the mean of the data), doesn't make sense in this context. Because the mean only affects the position of the data center, which isn't related to the direction of the mean difference. The second central moment yields similar conclusions when it is constrained with the assumptions made in the paper. However, in practice, we find the third central moment works much better. To some extent, the third central moment not only increases with the mean difference but also with the asymmetry of the two clusters. In backdoor attacks, the amount of benign data is usually much larger than poisoned data, which cannot be captured by the second central moment. That's why the third central moment performs better than the second central moment in our scenarios.

W3: The experiment is pretty insufficient. It only covers one datasets with only one poisoning rate. I suggest the author to give a more comprehensive experiments to show their proposed method's effectiveness. Some standard setting in https://github.com/SCLBD/backdoorbench is recommended.

Answer: Following the reviewer's suggestion, we conducted additional experiments on GTSRB. The results indicate that our method performs well in this context too. It's noteworthy that our attack with CLA on GTSRB didn't succeed, which is also the case in the BackdoorBench.