Tensor Train Decomposition for Adversarial Attacks on Computer Vision Models

1. The method proposed in this paper is not innovative enough.

(1) While the paper introduces the TETRADAT method for performing black-box adversarial attacks, it relies primarily on the PROTES optimiser, as described in the paper by Batsheva et al.[1], without making any innovative modifications to the PROTES algorithm itself.

(2) While this paper presents an approach that integrates a saliency map with the PROTES optimisation framework to perform adversarial attacks on deep neural networks, the concept of using saliency maps for such attacks is not new. Previous works, such as Liu et al.[2] and Dai et al.[3], have explored the use of saliency maps in the context of adversarial attacks.

2. There are some issues with the experimental section of this paper.

(1) Insufficient Benchmarking: The paper utilizes Onepixel, Pixle, and Square as baselines for comparison, all of which focus on perturbing a small number of pixels to achieve adversarial attacks. However, the paper lacks a comparison with query-based black-box attack methods, such as the approach presented by Bai et al. [4].

(2) Absence of Ablation Studies: The paper combines the Tensor Train-based methods with the attribution maps generated by auxiliary white-box models. However, there is a noticeable absence of ablation studies that demonstrate the effectiveness of incorporating the attribution maps. Ablation studies are crucial for understanding the incremental benefits of the components included in the proposed approach.

(3) Lack of Result Analysis: The paper presents results in Table 1, Table 2, Table 3, Figures 3, and Figure 4, but there is a lack of textual analysis to accompany these results. Moreover, Figures 3 and 4 seem to convey the same information, which may indicate a redundancy in the presentation of results. A thorough discussion of the results, including the implications and insights gained from the experimental data, is missing.

(4) Poor Experimental Results: The average L1 and L2 norms reported for the TETRADAT method in Tables 2 and 3 do not appear to demonstrate a clear advantage over the baseline methods. A discussion on why the TETRADAT method did not outperform the baselines in terms of these norms is necessary to assess the practical applicability of the proposed technique.

3. Some of the views are not supported by evidence.

The paper claims that TETRADAT produces the most realistic adversarial images (this becomes more noticeable as images are zoomed in), as stated in lines 496-500. However, this claim lacks evidence, as the authors have not provided any experiments with human observers to back up the assertion that TETRADAT's images are more realistically altered.

References

[1] Batsheva A, Chertkov A, Ryzhakov G, et al. PROTES: probabilistic optimization with tensor sampling[J]. Advances in Neural Information Processing Systems, 2023, 36: 808-823.

[2] Liu H, Zuo X, Huang H, et al. Saliency Map-Based Local White-Box Adversarial Attack Against Deep Neural Networks[C]//CAAI International Conference on Artificial Intelligence. Cham: Springer Nature Switzerland, 2022: 3-14.

[3] Dai Z, Liu S, Li Q, et al. Saliency attack: towards imperceptible black-box adversarial attack[J]. ACM Transactions on Intelligent Systems and Technology, 2023, 14(3): 1-20.

[4] Bai Y, Wang Y, Zeng Y, et al. Query efficient black-box adversarial attack on deep neural networks[J]. Pattern Recognition, 2023, 133: 109037.

The weakness of the paper is that the study focuses on untargeted attacks. It also needs to consider and compare with the targeted attack results. In adddition, the paper does not address if the approach is feasible for larger and deeper models.

• The authors classify TETRADAT as a query-based black-box attack method due to its reliance on querying the target model during the optimization process. However, since TETRADAT also uses an auxiliary white-box model to guide the attack, it shares characteristics with transfer-based methods, which typically require no queries to the target model. Given this similarity, a fairer and more comprehensive evaluation would involve comparing TETRADAT with transfer-based attack methods. This would provide insights into whether querying the target model yields significant advantages over transfer-based approaches, especially when both methods assume access to a white-box auxiliary model.

• While the paper fixes a query budget of $10^4$ for TETRADAT and other baseline methods, it does not provide a detailed comparison of the actual number of queries required to achieve a successful attack across different methods. Query efficiency is a critical measure of black-box attack performance, especially in scenarios where query access is limited or costly. Without a direct comparison of query counts, it is difficult to assess whether TETRADAT offers an advantage in terms of query efficiency relative to other black-box methods.

• TETRADAT relies on several hyperparameters, including TT ranks, perturbation amplitude, learning rate for the PROTES optimization, and the number of discrete perturbation levels. These hyperparameters require careful tuning to achieve optimal performance. The complexity of tuning these parameters makes the method less practical for deployment, especially for users without extensive experience in hyperparameter optimization. Additionally, there is limited analysis of how these hyperparameters affect the attack's effectiveness, which could have provided valuable guidance for setting these values.

• TETRADAT discretizes the perturbation levels for each pixel, simplifying the search space but potentially limiting the granularity of the perturbations. The chosen number of discrete levels influences both the subtlety and effectiveness of the attack. This discretization could restrict the method's flexibility to find the optimal perturbation, particularly in cases where finer adjustments are needed. Continuous optimization methods might perform better in scenarios requiring precise control over perturbation values.

• The evaluation is limited to ImageNet, lacking tests on additional datasets, which would demonstrate the method’s generalizability. A more diverse evaluation, including tests with varied hyperparameter settings, would offer a fuller understanding of TETRADAT’s robustness and effectiveness. Besides, a more detailed analysis of experimental results would be preferred.

• The presentation of this paper doesn't meet the ICLR standards and needs further improvement.

The experiment scope is too narrow.

Motivation is not clear.

Time complexity is missing.