DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness

"Probable bad params for attacks. The low success rates of attacks (especially GAMMA) might be due to a wrong initialisation. In the appendix, it is written that 200 as population size and query are used, but the number of queries for the GAMMA attack are computed as population_size * iterations. Also, the number of used sections is missing (which is a crucial point for the attack)."

We apologize for mentioning the query number as 200. We set the population size as 200, and ran it for 20 iterations. We have corrected it in the Appendix A.4.8. Also, for further confirmation, you can go to our provided code, and in the line 68 of the attack_malconv.py file, you can find the initialization. It is -

attack = CGammaSectionsEvasionProblem(section_population, CEnd2EndWrapperPhi(net), population_size=200, penalty_regularizer=1e-12, iterations=20, threshold=0.5)

"How you conducted the adversarial attacks? Which library did you use / how you obtained the code of attacks? Inside the provided material, it is not possible to test the attacks from Lucas et al."

For most of the attacks we used the secml-malware Python library. Also, we built the DRSM framework on top of the secml-malware library so that it can be easily reproduced, extended, and evaluated against more attacks in future (mentioned in the footnote of page 2).

We really appreciate your effort of checking our provided material and code. About Disp and IPR attack of Lucas et. al. – we collected the code implementation directly from the authors. Though we want to make everything of this work publicly available, we cannot do that for Disp and IPR attack. The white-box implementation of these attacks was kept private by the authors, and before access, we agreed to keep it private too.

"False statements. The authors state that "it is difficult to add more than 10% of content". This statement is false, since adversarial malware attacks are automated through tools. Papers like [Demetrio et al. 2021a&b / Lucas et al. 2021] can increase the size more than 10% of the file size (Lucas et al. bound it to 5% just to not enlarge too much the input file)."

In the last paragraph of subsection 6.2, we mentioned that – perturbing 200KB in 2MB file (=10%) is ‘challenging’ in a malware file. If that is the concern, we have rephrased it and toned it down to ‘considered as a sizeable modification’. If you have further suggestions, let us know.

"Lastly, Header Modification is not proposed by Nisi et al. 2021, but it is contained inside the SecML Malware library, inspired by the paper (that states which fields are not used by the loader anymore)."

We are aware of the fact that – Nisi et. al. 2021 did not explicitly propose the header field modification attack but we wanted to mention the most relevant paper to the attacks so that interested readers can study them. We have changed ‘proposed by’ to ‘motivation from’ in our text for Nisi et. al. paper. If you have further suggestions regarding presentations, we are more than happy to incorporate them.

"Dataset concerns. While the release of a goodware dataset is for sure a great contribution, I am doubtful on the composition of such corpus. In particular, the sources might contain malware or generic unwanted propgrams (Softonic is known to host plenty of installers and grayware that asks you to install other third-party programs). The authors should better clarify the origins of these data, or at least try to study the quality of the provided ground truth. Otherwise, the dataset might contain biases that reduce the fairness of the publication."

Evaluating the quality of goodware data is a very good point, and we are actively trying to address this. We are in the process of getting access to the premium API of VirusTotal, and are planning to scan all the benign files to filter out any malicious file before the official release. Meanwhile, we are working on getting some preliminary statistics before the discussion ends.

Thank you for your concise and constructive comments.

"No white-box evaluation. The authors state that they compute attacks on the base model, thus framing them as white-box attacks (like Partial DOS; Shift, etc). However, these are evaluated as black-box transfer attack, and thus should be clarified on the paper."

Thank you for pointing this out. We agree the current description might be confusing: Since the attacks require white-box access to the base models, they are indeed white-box. However, we agree they are in some sense transfer attacks since it is the base model rather than the DRSM itself being attacked. We have updated the Section 7 for clarity. Let us know if you have further suggestions regarding presentations. We appreciate your feedback.

*"Limitations and related work not addressed. The paper does not discuss limitations of their methodology, by just saying that it is certification is a difficult problem to solve.

We wanted to put a ‘Limitations’ section before the ‘Conclusion’. However, due to the page constraint, we could not do that. So, we have added a ‘Limitations’ section in our Appendix (A.6) and referred to it in our ‘Conclusion’ section.

Also, related work misses a preliminary (but unpublished) paper [1] that addressed the problem in the early months of 2023 (more than 6 moths ago). It would be better to mention the fact that preliminary work on certification for malware detection are already there. [1] Certified Robustness of Learning-based Static Malware Detectors - https://arxiv.org/pdf/2302.01757.pdf"*

Although ICLR policy tolerates such overlooks for preprints without peer reviews that are published after May 28 (https://iclr.cc/Conferences/2024/ReviewerGuide), we have discussed this recent paper in the ‘Related Work’ of our revision.

Thank you for your work during this rebuttal period. I raised my score to ACCEPT (8).

"One issue would be with attacks which INSERT bytes. Section 3.2 seems to suggest this is possible ("attacker can modify or add any bytes in a contiguous portion"). Surely, if added at the start of the file, this can change the contents then of EVERY window, as all the bytes get shifted to the right. So adding bytes does not seem to be in the threat model that would give certified robustness."

"In fact, the above comment may explain why the DOS Extension attack has such a big effect on the DRSM models in Figure 5."

This is a very good observation, and we are thankful for this. It might be the case that – the DOS extension attack has an impact on the rest of the windows to some extent, and thus the DRSM-4 and DRSM-8 are less robust to this attack. Since this attack still cannot directly perturb or alter bytes in other windows, it is partially aligned with our threat model. So, we have modified our Table 4, subsection 3.2, and Appendix A.4.3, accordingly. We have also added a subsection in the Appendix (A.4.1) discussing the probable reason for the higher ASR of DOS Extension Attack.
A future mitigation can be – extracting sections from the file and training different base classifiers on each of them (discussed in the 'Limitations' in Appendix A.6). Thus, attacks like DOS extension will not be able to impact other (or later) windows.

"Would it be possible to indicate the values of \Delta on the x-axis of figure 3? Everything has been in terms of that up to this point."

We have added another figure (Figure 10 in the Appendix A.4) that shows the certified accuracy in terms of $\Delta$. From this figure, it is interpretable that a smaller window size achieves higher certified accuracy. At the same time, we want to mention that – a smaller window allows attackers a smaller budget for perturbation. For example, if $\Delta =2$, in DRSM-4, the attacker can perturb up to 511K bytes, whereas in DRSM-24, it is 82K bytes. So, it might not be fair to compare these models just depending on the $\Delta$, and hence, we kept the Figure 3 showing accuracy with respect to perturbed bytes in the main text, and put the Figure 10 in the Appendix so that interested readers can see them and easily interpret the results. We would also love to have your feedback on this one.

"There are a few grammatical issues throughout the paper - it needs a polish before publication,"

We really appreciate your effort in reading our paper thoroughly and pointing out the grammatical issues. We have corrected all of the mentioned ones. You can find them in blue text in the updated version of our submission. Let us know if you find anything else. We would love to have your feedback.

"It would have been good to see a description of de-randomized smoothing in the related work, as this is core to the idea."

While we understand that it would have added more clarity if we discussed de-randomized smoothing more in the ‘Related Work’, we could not do it due to space constraints. So, we have added a subsection in the Appendix (A.2) discussing the de-randomized smoothing in-depth, and have referred to this in our ‘Related Work’.

"X \subset [0,N-1] is wrong. [0,N-1] is a real-valued continuous interval. I think it means X is the set {0, 1, ..., N-1}"

Thank you for pointing this out. We have addressed this in our revised version.

"The paper claims to be “first to offer certified robustness in the realm of static detection of malware executables”. However there is prior work on this topic by Huang et al. (2023) which appeared on arXiv in January 2023. Their work considers a different threat model for malware: edit distance robustness rather than patch robustness."

Although ICLR policy tolerates such overlooks for preprints without peer reviews that are published after May 28 (https://iclr.cc/Conferences/2024/ReviewerGuide), we have acknowledged and discussed this recent paper in the ‘Related Work’ of our revision. Additionally, we want to point out that – along with the different threat model (that you already mentioned), this paper adapts the randomized smoothing scheme, which differs from ours.

We really appreciate your effort in going into the details and for your constructive feedback.

"The paper can be seen as applying an existing method (de-randomized smoothing) to a new domain (malware). Although the authors claim that it is “challenging” to adapt de-randomized smoothing for malware, I’m not convinced. The proposed method is a form of structured ablation, originally studied by Levine & Feizi (2020) for 2-d inputs with homogeneous base classifiers. The modification from 2-d to 1-d inputs and from homogeneous to heterogeneous base classifiers seems straightforward. Moreover, the proposed method has appeared in prior work in a more general form by Hammoudeh & Lowd (2023)."

We respectfully argue that the success of a ‘general’ method in a specific domain should not be taken for granted. An example will be the paper of Hammoudeh & Lowd (2023) mentioned in your review, which also resembles Levine & Feizi (2020): One contribution of Hammoudeh & Lowd (2023) is their study of partitioning strategies, in which they suggest using stridded input dimensions (pixels) is beneficial for vision tasks, potentially because information contained by adjacent pixels are redundant to some extent. In contrast, for the malware domain, due to its nature, some adjacent bytes are malicious only when they are combined, i.e., not considering adjacent bytes might end up representing a whole different instruction (or invalid instruction) that might not be malicious.

"Section 2 states that it’s “surprising” MalConv is still considered state-of-the-art for malware detection on raw byte sequences, given it was released in 2018. The authors attribute this to limited availability of public data. However, I think the main reason is due to difficulties in scaling more complex models (such as transformers) to very long sequences, containing upwards of a million tokens. It’s worth pointing out that the authors of MalConv have released a follow up model known as MalConv 2 (Raff et al., 2021)."

Thanks for pointing this out. Initially, we did not include this due to space constraints, but we have modified the first paragraph in the 'Related Work', and discussed the MalConv 2 model.

"Section 3 states that the input vector fed into the network “has to be of a fixed dimension”. This is not true in general, and I don’t believe it’s true for MalConv. I believe it is possible to support arbitrary length inputs in modern frameworks such as PyTorch by specifying None for the size of the dimension."

With that sentence, we wanted to mention the original implementation of MalConv that was done for 2MB of file size.

"Section 3.1 states that models like EMBER and GBDT “can work only on feature vectors”. I’m a bit puzzled by this statement. When composed with their feature extractors, these models must be able to operate on raw binaries, otherwise they would be useless as malware detectors?"

We are aware that EMBER, GBDT can work as malware detectors that require an extra feature extraction step unlike our work, and we apologize for the misunderstanding. We have rephrased that sentence in subsection 3.1.

"Section 5 states that vision-oriented ablation techniques such as masking and block ablations are infeasible for byte sequences. However I can’t see why these wouldn’t work on 1d sequences? It seems feasible to mask bytes or ablate 1d blocks."

An instruction is converted to multiple bytes that are usually contiguous, and masking a random byte might end up changing the meaning of an instruction or in the worst case, an invalid instruction. Block ablation was proposed for 2D input, it is not clear how it can be applied for 1D input. (We also update our manuscript with a recap of de-randomized smoothing in vision tasks in Appendix A.2.).

"Section 7 states MalConv NonNeg “has been believed as a robust model for a long time”. It would be good to include a citation for this claim."

We see how this statement might be controversial. We have removed it from section 7.

"It’s great that the authors are planning to release the PACE dataset. However, the current description of the collection process is a bit light on detail. It would be helpful to describe how binaries were selected from the various sources. For instance, was there a preference for recent binaries? Are the binaries for a single platform (e.g., Windows x64) or multiple platforms? How do you know the binaries are benign?"

Thanks for appreciating our plan to release the PACE dataset. The sources of this dataset are given in Table 2, and we crawled these websites to download the benign files. We downloaded them in August, 2022 and there was no preference for recent binaries. Yes, the binaries are for a single platform (windows). Evaluating the quality of goodware data is a very good point, and we are actively trying to address this. We are in the process of getting access to the premium API of VirusTotal, and are planning to scan all the benign files to filter out any malicious file before the official release. Meanwhile, we are working on getting some preliminary statistics before the discussion ends.

"I found the description of the threat model and certificate unclear. Section 3.2 states that the attacker is allowed to “modify or add any bytes in a contiguous portion”. I understand that “modify” means overwrite or replace, but it’s not clear what “add” means in this context. For instance, “add” could mean “insert” or “append”, or it may mean “increment or decrement by some amount”. The mathematical description x′=x+delta implies the original sequence x is additively perturbed by delta (which is undefined), but this seems to be at odds with the earlier description. Reading between the lines, my understanding is that the certificate covers a contiguous chunk of the original file being overwritten, which may include some bytes being appended to the end of the file. This should be precisely stated somewhere. The current definition of the certificate in Section 5 is in terms of ablated sequences – it would be helpful to translate this to the input space."

Thanks for your constructive feedback and asking for the clarification of the term ‘add’. We have rephrased a few sentences in the second paragraph of our threat model (subsection 3.2). With the term ‘add’, we meant the attacker can ‘insert’ or ‘append’ any bytes; not ‘increment / decrement’. Additionally, we have mentioned that it has to be bounded in a contiguous portion. Also, we apologize if the mathematical description looked confusing. We have reformed that into a simple sentence now for better clarity. Additionally, we have rephrased a few sentences in the third paragraph of the threat model, and mentioned what type of attack falls within our threat model. Let us know if you have further suggestions about the presentation. We would appreciate your feedback.

"A characteristic feature of the malware domain is that inputs vary in length. However, it’s not clear to me how the proposed classifier architecture handles this. As an example, consider a 100 KB malicious file and a classifier with a maximum input length of 2 MB. For n in the range 4–20, the malicious file fits within a single block, meaning it is passed to a single base classifier, while the remaining n-1 base classifiers receive padding as input. Assuming the base classifiers predict “benign” for padding, the votes are 1 “malicious” and n−1 “benign” giving a prediction of “benign”. I wonder if I’m missing something here, because it seems the classifier is guaranteed to make false negative errors on small files, which are abundant according to Figure 6."

This is a very good observation. To tackle this issue in our DRSM framework, we consider the votes (or predictions) from the base classifiers that get any input (except padding). We could consider all votes and solve this by adding an extra learnable layer (such as logistic regression) on all votes, but it would have hurt the non-differentiability property (eventually, certified robustness) of the whole framework.

"The proposed "window ablation" strategy seems to be applicable only to scenarios where perturbations are clustered together (similar to patch attacks in the CV domain). However, many adversarial attack methods for malware disperse the inserted perturbations across various locations within the software. It remains unclear whether the proposed method would still be effective in such cases."

We really appreciate your concise review.

Yes, you are right. The proposed method is ‘theoretically’ for the attacks that can modify or add in a contiguous region. And we also agree that there are many adversarial attacks that do not follow this and can modify/add in multiple regions. So, in this work, we actually evaluated our proposed model DRSM against such attacks, because we believe that – only ‘theoretical robustness’ is not enough for a security-critical application like malware detection. In section 7 ‘Empirical Robustness Evaluation’, we considered in total of 9 attacks where 5 of them can modify multiple regions in a file. In Table 4, we listed all these attacks with their short description and alignment with our threat model. For example, Slack append, Header field modification, and Gamma attack can perturb multiple regions in a malware file, and recently proposed Disp and IPR attacks can modify a file on the instruction level and are not limited to a certain region.

We used these attacks to compare our DRSM with the baseline model ‘MalConv’ and its more robust variant ‘MalConv (NonNeg)’. The results are shown in Figures 4 and 5.

"It is essential to provide a comparative analysis of the author's proposed defense method against existing adversarial example defense techniques for malware."

The existing defenses for adversarial malware can be broadly divided into two – Non-negative classifier, and Adversarial training. We have already included the former one, ‘MalConv (NonNegative)’, in our work and compared it in terms of standard accuracy, certified accuracy, and empirical robustness in Table 3 and Figures 4, 5 ( and Appendix A.3). We also want to emphasize that – recent work by Lucas et. al. [1] has already shown that – in most cases, the adversarially trained models (the second defense) do not provide good robustness against other attacks. Moreover, such defense compromises the standard accuracy too, for example, Lucas et. al. showed training the model adversarially on Kreuk-0.01 degraded the true positive rates to 84.4% ~ 90.1%. We have discussed this in the second paragraph of section 2 (Related Work).

[1] Adversarial training for {Raw-Binary} malware classifiers. https://www.usenix.org/conference/usenixsecurity23/presentation/lucas

"The construction of many malware samples follows a piggyback approach, where the majority of the software consists of benign code, with only a small portion exhibiting malicious behavior. That’s to say most of the ablated sequences will be given a benign label. The algorithm proposed by the author may result in false negatives for such malware samples."

This is a very good observation, and it is true that most of the contents in a malware file are actually benign. This argument actually aligns with our result too. We found that – if we increase the number of windows (decreasing the length of ablated sequences), the probability of an ablated sequence getting classified as ‘benign’ gets higher because a smaller ablated sequence would cover less content, and hence, less/no malicious content. As a result, for the higher number of windows ($n$), DRSM models have less standard accuracy, and we showed this in Table 3 and discussed this in subsection 6.1. For example, DRSM-4 has 98.18% standard accuracy, whereas DRSM-24 achieves 90.24%. The goal of this paper was to – find a balance between standard accuracy and robustness.

"Is the winning class of Fig. 2 is "benign"?"

Figure 2 is a generalized Figure where the input file can be anything (malware or benign). The green check stands for a correct prediction, whereas the red cross stands for a wrong prediction. At the end, the bars stand for the number of correct vs wrong predictions. For example, let us assume that the input file is an adversarial malware, and the perturbed region of that malware falls into one of the ablated sequences of DRSM (shown with a red small block in ablated sequences). So, the base classifier misclassifies that sequence (shown with a red cross) but classifies the rest of the sequences correctly (shown with green checks). Hence, the winning class is still ‘malware’ (shown with the green bar in ‘count class prediction’). We apologize if the figure is confusing. We have added a small description in the caption. If there is any scope to make the presentation better, let us know. We would love your feedback.

"The rationale behind the design choice is unclear. Why have the authors chosen MalConv as the baseline classifier? There are numerous alternative models that can serve as the baseline classifier, such as LGBM, RF, and SVM. The authors should consider evaluating their framework with these alternative baseline classifiers."

We want to emphasize that CNN-based models like MalConv can take the whole raw bytes as input whereas models like LGBM, RF, or SVM require feature engineering (e.g., byte entropy), and they cannot take raw bytes as input. These features are known to be specifically targeted by adversaries to avoid detection, which makes "'being able to take the whole raw binary" attractive. However, this new capability introduces a risk of adversarial attacks too, which motivates us to position our defense for CNN-based models. As MalConv is one of the state-of-the-art CNN based static classifiers, it was a suitable choice to evaluate our framework. However, note that our window-ablation scheme is agnostic of the base classifier as it operates at the input level. As a result, it could be applied to any detector that consumes raw bytes.

"The absence of comparisons with a broader range of real-world antivirus engines accessible via VirusTotal is notable. It would be beneficial, for instance, to determine how many antivirus engines effectively identify the malware and test cases as malicious."

"Another deficiency is the absence of specific details regarding the process of compromising executable files. The authors should provide a more comprehensive explanation of how to generate adversarial examples within the problem-space[Ref-1], which encompasses defining a comprehensive set of constraints on available transformations, preserving semantics, ensuring robustness to preprocessing, and maintaining plausibility."

The main focus of this paper was to propose a better defense model for adversarial attacks, not the attacks themselves. Moreover, we evaluated our models against 9 different attacks and there is a page limitation. So, we just kept the gist of the attacks in our main paper. Table 4 lists all 9 attacks with their short description, and alignment with our threat model and settings. Additionally, in Appendix A.4, we included details for all of these attacks and their implementation in this work. Also, all of these attacks have already been proposed in published prior works showing that they do not compromise executable files, and we cited them so that any specific details about them can be retrieved if necessary. And some of these attacks (such as Disp, IPR, GAMMA, etc.) had already been tested against VirsuTotal and were found successful. We also want to mention that -- Disp, IPR attack (Lucas et. al. (2021)) reported that they can evade VirusTotal in 49%-53%, whereas in DRSM, these attacks could evade 42% and 9.50% cases, respectively, even for our weakest model (DRSM-4).

"In my view, there appears to be a contradiction between Figures 1 and 2. As discussed in Section 5, malicious ablated sequences are expected to influence their respective base classifiers, and the predicted winning class should be "benign." However, in Figure 1, the predicted winning class is labeled as "malware,". It confuses me. If this work indeed presents a more robust framework that prevents attackers from generating adversarial examples, then the winning class should ideally be "benign." However, if the winning class is consistently benign, it suggests that the proposed framework might miss detecting certain malware instances, creating a contradiction."

“If this work indeed presents a more robust framework that prevents attackers from generating adversarial examples, then the winning class should ideally be "benign."” – we want to mention that – this work does not prevent attackers from generating adversarial examples. We proposed a framework (DRSM) on top of an already existing classifier that can improve its robustness for adversarial malware; our goal was not to stop malware authors from generating them.

In Figure 1, we tried to show the fundamental difference between the original base classifier (MalConv) and our method (DRSM) with a toy example in a simple way. Here, for DRSM, the adversarial malware gets ablated into 3 non-overlapping windows (or sequences) and generates 3 different predictions. Since the perturbation impacted only one window (the middle portion in the file written with red font), DRSM gave the wrong prediction (benign) only for that one. But for the rest of the windows (the first and third portion in the file written with black font), DRSM correctly classifies them as ‘malware’. As a result, ‘malware’ (2) wins against ‘benign’ (1) in voting, and DRSM gives the final output as ‘malware’.

"Furthermore, the authors should establish the relationship between the window size (w) and the resulting certified robustness. For example, does a smaller window size lead to improved certified robustness?"

Thanks for asking about the relationship between window size and certified accuracy. Yes, a smaller window size leads to a better certified accuracy. And we discussed this in the last paragraph of subsection 6.2. We mentioned that – “By analyzing Table 3, we can see that $n$ has a positive and negative correlation with certified and standard accuracy, respectively. While DRSM-24 provides the highest certified accuracy (53.97%), it has the lowest standard accuracy (90.24%).” The Table 3 shows the certified accuracy for the same $\Delta (=2)$ for each model variant, and notably, DRSM-24 (with the smallest window size) achieves the highest certified accuracy (53.97%) whereas DRSM-4 (with the highest window size) achieves the lowest certified accuracy (12.2%). Additionally, we have added another figure (Figure 10 in the Appendix A.4) that shows the certified accuracy in terms of $\Delta$. From this figure, it is interpretable that smaller window size achieves higher certified accuracy. At the same time, we want to mention that – a smaller window allows attackers a smaller budget for perturbation. For example, if $\Delta =2$, in DRSM-4, the attacker can perturb up to 511K bytes, whereas in DRSM-24, it is 82K bytes. So, it might not be fair to compare these models just depending on the $\Delta$, and hence, we kept the Figure 3 showing accuracy with respect to perturbed bytes in the main text, and put the Figure 10 in the Appendix. We would appreciate your feedback too.

"Insufficient theoretical analysis of certified robustness when facing multiple malicious ablated sequences. Specifically, the authors assume that an attacker generates a byte perturbation of size p and can modify a maximum of ⌈p/w⌉+1 ablated sequences. However, if the attacker simultaneously inserts multiple adversarial code segments at different locations, how will this impact the certified robustness? The authors should offer a more in-depth theoretical analysis of this scenario."

Thanks for such a detailed response. We appreciate your effort in reading our paper thoroughly.

As the de-randomized smoothing approach was borrowed from the computer vision domain, we followed the same threat model where the attacker can add a patch of a specific size (add or modify byte sequence of $p$ size, in our case). In our theoretical study, we wanted to keep it as similar as the CV domain. At the same time, we also understand that the malware domain does not work like that, and your point on – "the attacker simultaneously inserts multiple adversarial code segments at different locations" is totally valid and we agree with that. Therefore, we empirically evaluated the robustness of our DRSM approach against such attacks that can insert multiple adversarial code segments (section 7). To be specific, we have included attacks, namely Slack Apppend, Header Field Modification, Gamma, where the attacker can include or modify at multiple places, and stronger attacks, namely Disp, IPR, where the attacker can even modify on the instruction level. Table 4 includes the list of attacks we experimented with in this paper where the column ‘Threat Model’ indicates their alignment with our threat model. In Figures 4 and 5, we showed the attack success rates for these attacks. Also, we have included how all these attacks work and were implemented in Appendix A.4.