CAN DEEPFAKE SPEECH BE RELIABLY DETECTED?

Vulnerability Already Well-Known:

Deep learning-based audio models are already known to be susceptible to evasion attacks [1], with extensive literature on optimization-based and signal processing-based attacks. In fact, the signal processing family of attacks might be even better suited than optimization based ones (like PGD and iFGSM as used in the paper) since they do not even require query access to the model. At this stage, the community would benefit more from methods to enhance robustness in deepfake detection. To improve the paper, the authors could propose, implement, and evaluate new approaches to strengthen deepfake detection systems based on insights from this study.

Lack of Information on Human Study:

The authors do not provide enough details to assess their human study adequately:

• Did they obtain IRB approval before conducting the experiments?
• How many participants were involved in the study?
• How was the study structured?
• What was the null hypothesis?

Reference:

[1] The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems

As this paper asks an evaluation question, it needs to back it with substantive evaluations. I suggest the following improvements:

• The complete evaluation is performed on 100 samples from 3 text-to-speech (TTS) datasets. Essentially basing all analysis on 300 samples. There are newer speech-to-speech datasets, such as DECRO [1]. Moreover, audio deepfake datasets are easy to create. If we have 100 speakers then we can create 100 * 99 (9,900) by inferring a speech generator, such as FREEVC, QuickVC, Suno-AI's Bark, TorToiSE, CoquiTTS. Subsequent filtering to retain high quality sample should be explored.
• If the datasets were in any form skewed, randomly sampling from them would carry forward those biases. A demographically and phonetically-diverse set of samples would give representation to practical edge cases.
• While white-box access to commercial deepfake detectors is harder to obtain, include them into black-box and model agnostic evals. Examples are resemble.ai, elevenlabs.io. In such case SimBA might not work, as confidence score is generally not given.

A general lack of discussion:The paper lists its insights but fails to make readers confident to believe what they are reading. For example, In 3.3, "ASSIST-L, being smaller, may form smoother decision boundaries, leading to better generalization and robustness against perturbations." First, the language is hand-wavy as no evidence is given, and second, this insight itself is surprising.

[1] Transferring Audio Deepfake Detection Capability across Languages, WWW'23. https://github.com/petrichorwq/DECRO-dataset

1. The paper only evaluates the robustness of four SSDs against adversarial attacks, which may limit the overall scope and depth of its findings. More advanced SSDs should be incorporated.
2. The evaluation may not be sound. For example, in section 3.2, the paper claims that "Deepfake speech from TTS not seen during training is more likely to bypass SSDs.". However, it does not provide the benign detection performance of the SSDs across different datasets, which is crucial as existing SSDs often exhibit poor generalization across datasets. If the SSDs trained on ASVSpoof2019 already perform poorly on, for example, the In-the-Wild dataset, then the observed attack success rates cannot be directly attributed to the effectiveness of the attack itself. This also causes confusion in the transferability experiments, where the paper finds that some SSDs are extremely vulnerable on out-of-domain data.
3. Additionally, the human evaluation is insufficiently rigorous, and the experimental design lacks essential details. For instance, the paper does not specify the number of human subjects involved or the procedures used in conducting the experiments. These methodological details are important for assessing the validity of the human evaluation but are absent from the paper.
4. The study does not examine proprietary or commercially deployed SSDs, which could exhibit varying levels of robustness compared to open-source models. In addition, it would be also interesting to see if the adversarial audios crafted by open-sourced SSDs can transfer to commercially deployed SSDs.
5. While the paper effectively identifies vulnerabilities in SSDs, it does not provide a thorough examination of potential improvements or specific countermeasures that could help address these issues.
6. The insights presented in this paper are somewhat limited and largely familiar to the adversarial machine learning community.
7. Lack of details and code for reproducibility.

1.Lack of Novelty: The challenges related to deepfake audio detection, particularly the decline in performance across domains, are well-established in the literature. The vulnerability of detection models to adversarial attacks is also widely recognized. Therefore, dedicating an entire paper to simply illustrating these issues does not contribute new insights to the field.

2.No Technical Contribution: The paper primarily applies classic adversarial attack methods to examine the vulnerability of detection models, without offering any novel approaches or techniques.

3.Insufficient Detail on Model Training: The paper lacks sufficient detail regarding the training process of the detection model. For example, was data augmentation employed to enhance the model’s generalization ability? There are various techniques that can improve model robustness, such as adding white noise to training data, which has been shown to increase resistance to adversarial attacks. The authors could investigate such approaches, perform a more comprehensive evaluation, and derive more valuable insights.

4.Poor Paper Organization: The paper’s organization could be improved. For example, figures are often located far from the sections that describe them, making it difficult for readers to follow the flow of the paper.