A Novel Security Threat Model for Automated AI Accelerator Generation Platforms

Thank you for your valuable suggestions! We have addressed the concerns you raised below.

W1. The preliminary section does not provide any information about the prior works on bit-flipping attacks. The lack of discussion makes it hard to understand the real innovation of this work since previous attacks have explored how to identify the sensitive kernels in the pre-trained quantized model (e.g., ProFlip).

Thank you for raising this important concern.

In the paragraph starting at line 242 of the manuscript, we have discussed existing bit-flipping attack methods such as BFA and T-BFA, and provided actual comparison results in Section 4.3. In these paragraphs, we not only review the main objectives and strategies of existing bit-flipping attack methods but also thoroughly analyze their limitations, particularly emphasizing that prior works do not adapt well to HT designs. Building on this analysis, we present our innovation: the proposed C-SFE algorithm, which is better suited for HT design and offers enhanced stealthiness.

We chose not to include detailed implementation specifics of prior works (such as mathematical formulas) in the preliminary section because our proposed C-SFE algorithm does not rely on these works. Instead, the proposed algorithm is a completely new attack scheme specifically tailored for HT, demonstrating independent innovation. Therefore, detailing the implementation aspects of prior works does not substantially aid in understanding our work and may instead make the content appear redundant.

W2. Why we cannot directly apply the existing bit-flipping attacks on AI accelerators. The challenges need to be clarified and empirical results can be used to support this.

Thank you for your insightful question.

In the paragraph starting at line 242 of the manuscript, we explain why we cannot directly apply existing bit-flipping attacks on AI accelerators. This is because current methods adopt an attack model combining adversarial weight attacks with physical attacks on DRAM (e.g., T-BFA + Row Hammer attack). In these methods, the constraint is that the fewer bits of the model parameters are flipped, the better. However, our proposed threat model is based on adversarial weight attacks combined with a HT (i.e., C-SFE + HT). Compared to the previous attack model, this approach also needs to consider the hardware resource consumption of the hardware Trojan. Therefore, we need to add additional constraints on the regularity of the attack locations and the number of parameters in the adversarial weight attack (in the experimental part of the manuscript, the model uses 8-bit quantization, so each parameter consists of 8 bits). In Section 4.3, we compare the proposed attack method with existing methods. The results show that existing attack methods cannot be directly combined with hardware Trojans.

W3. The design of Cross-layer Sensitive Filter Exploration (C-SFE) is not clear. The authors mentioned in Section 3.4 that 'CSFE expands the decision boundary of a targeted category by flipping a few bits in one kernel per layer'. However, the paper does not explain whether different layers are equally important and whether the attack can exploit a single layer only.

Thank you for your comments. Our responses are provided below for each point.

Whether different layers are equally important ?

During the development of our attack algorithm, we discovered that for N-to-1 attacks, different target classes require attacking different layers. Therefore, the significance of each layer depends on the target class. Furthermore, we would like to clarify that in the proposed algorithm, the number of kernels and parameters attacked for each layer are identical (a kernel consists of 3×3 parameters). This regular constraint is designed to accommodate the design of hardware Trojans.

Whether the attack can exploit a single layer only ?

During the algorithm design process, we discovered that implementing an N-to-1 attack is achievable by targeting a single layer. However, compared to attacking multiple layers, it requires a larger number of parameters to be compromised. For example, using ResNet-18 as shown in Table 1 of the manuscript, a single-layer attack approximately requires compromising 40 parameters, whereas a cross-layer attack only needs to target 12 parameters (4 per layer). Since most AI accelerators operate using Systolic Array architectures, the hardware Trojan design for a single-layer attack would need hardware resources capable of locating and modifying these 40 parameters. In contrast, a cross-layer attack only requires hardware resources to locate and modify 4 parameters. Therefore, the latter approach utilizes fewer hardware resources. Considering these two points, we abandoned the single-layer attack scheme during the design process.

Dear Reviewer vWaG,

As the discussion period is nearing its conclusion, we kindly ask if you could review our response to ensure it addresses your concerns. Your feedback is greatly appreciated.

Thank you for your time!

Best,

Authors

Thank you for your valuable suggestions! We have addressed the concerns you raised below.

W1. The threat model is vague and unclear on some assumptions. For example, sections 3.1 and 3.2 are confusing regarding what is the goal of the attacker. Is it a hardware trojan attack or an adversarial weight attack? Both terms are used. However, the threat model assumption is not equivalent for both cases. Moreover, there is discussion on how the attacker would gain access to model parameters, but hardly any discussion on how they will modify the parameters. The only discussion in the paper states in line 194, but how an attacker would insert this malicious code.

Thank you for your comments. Our responses are provided below for each point.

Sections 3.1 and 3.2 are confusing regarding what is the goal of the attacker. Is it a hardware trojan attack or an adversarial weight attack? Both terms are used. However, the threat model assumption is not equivalent for both cases.

We believe that the terms "adversarial weight attack" and "hardware Trojan" do not conflict for the following reasons:

1. An adversarial weight attack primarily refers to an algorithm at the software level, aiming to identify sensitive bits or parameters within the model. When executing the attack, methods such as Row-Hammer Attack (RHA) are employed to flip data bits in DRAM. Most prior works have adopted such approaches (i.e., adversarial weight attack + RHA), as exemplified by BFA and T-BFA introduced in line 243 of the manuscript.

2. Our proposed threat model employs an adversarial weight attack combined with a hardware Trojan (HT) (i.e., adversarial weight attack + HT), presenting an innovative approach to uncover security vulnerabilities in automated AI accelerator generation platforms that have been widely studied but lack sufficient security focus. In the paragraph starting at line 184 of the manuscript, we provide a detailed explanation of how the adversarial weight attack collaborates with HT, which does not conflict with the threat model introduced in Section 3.1.

There is discussion on how the attacker would gain access to model parameters, but hardly any discussion on how they will modify the parameters.

The paragraph associated with line 194 in the manuscript primarily introduces how the proposed threat model attacks designs that adhere to the AI accelerator design paradigm. This outlines a general attack process and, therefore, does not delve into implementation specifics. As a concrete implementation, we demonstrate this attack process using Gemmini as an example in Section 3.3. Additionally, in line 237, we provide a detailed explanation of how HT modifies the parameters in conjunction with Figure 2(b). Specifically, when the Trigger is activated, the Payload's task is simple address comparison and data overwriting.

How an attacker would insert this malicious code ?

The hardware Trojan consists of fixed code embedded within the accelerator template. For the implementation of the HT, please refer to the Trigger design (see line 1196 in the LoopConv.scala provided in the anonymous GitHub link in the manuscript) and the Payload design (see line 544 in Scratchpad.scala).

W2. The most concerning part is it is not clear whether the attacker can access the training stage or not. If not, then how can they modify model parameters? In the model production stage and inference stage where exactly the attacker is located.

Thank you for your comments. Our responses are provided below for each point.

Whether the attacker can access the training stage or not?

Our answer is NO. As illustrated in the manuscript (starting from line 145), we emphasized in the threat model that the information accessible to the attacker is limited to the model structure, model parameters, and a small set of images used for model quantization calibration. However, hyperparameters, optimizers, and training datasets used during the model training phase are not included. Therefore, the attacker cannot access the training stage.

If not, how can attacker modify model parameters?

The functionality of the proposed C-SFE is not to further optimize the accuracy of the user-input model but rather to explore which parameters in the model are sensitive to specific classes. Therefore, even without access to information from the training phase (e.g., training dataset), C-SFE can still identify the model's sensitive parameters based solely on a small subset of the dataset used for quantization calibration. Additionally, AI accelerator generation platforms have the authority to modify the model; for instance, during the quantization phase, they need to read the model parameters, calculate the quantization scale, and rewrite it into the model. These aspects align with previous studies (e.g., BFA, T-BFA).

W4. Attack algorithm is naively adopting a genetic algorithm. Eqn. 1 is very similar to computing gradient, since the attacker can access model parameters why not compute the gradient. The authors claim their method does not need gradient computation, more justification is needed why GA would be faster than computing gradients.

Thank you for your comments. Our responses are provided below for each point.

Eqn. 1 is very similar to computing gradient.

We believe that Equation 1 is fundamentally different from computing gradients. Equation 1 calculates the cumulative difference in the model's output confidences between the original parameters and the modified parameters across all inputs. This method assesses the overall impact of parameter changes on the model's predictions. In contrast, computing gradients involves calculating derivatives of a loss function with respect to the model parameters, which is used for optimization during training. Since our focus is on evaluating the changes in output distributions rather than optimizing the parameters, computing gradients is not applicable in this context.

Why not compute the gradient ?

In the paragraph starting at line 242 of the manuscript, we explain why we cannot directly apply existing gradient-based attack methods on AI accelerators. This is because current gradient-based attack methods adopt an attack model combining adversarial weight attacks with physical attacks on DRAM (e.g., T-BFA + Row Hammer attack). In these methods, the constraint is that the fewer bits of the model parameters are flipped, the better. However, our proposed threat model is based on adversarial weight attacks combined with a hardware Trojan (i.e., C-SFE + HT). Compared to the previous attack model, this approach also needs to consider the hardware resource consumption of the hardware Trojan. Therefore, we need to add additional constraints on the regularity of the attack locations and the number of parameters in the adversarial weight attack (in the experimental part of the manuscript, the model uses 8-bit quantization, so each parameter consists of 8 bits).

More justification is needed why GA would be faster than computing gradients.

We would like to clarify that in our manuscript, we did not claim that the Genetic Algorithm is faster than computing gradients.

The original intention of this manuscript is to reveal the lack of security research related to the AI accelerator design paradigm by proposing a threat model. Since there has been no prior research combining adversarial weight attacks and hardware Trojans, as mentioned in line 360 of the manuscript, our proposed C-SFE algorithm is merely a feasible solution that satisfies this threat model; it cannot guarantee optimality but can serve as a baseline for future related research.

W5. Figure 7 is confusing; the description is not clear on the conclusion. Both attacks target the middle three convolution layers, but what does that mean from an attack efficacy and efficiency perspective?

The primary aim of Section 4.3, which corresponds to Figure 7, is to highlight the suitability of our proposed C-SFE method for hardware Trojan implementation (as stated in the first sentence of Section 4.3), particularly in contrast to previous attack algorithms like T-BFA.

To address the confusion regarding Figure 7:

1. Attack Efficacy: In line 457, we emphasized that both attacks achieve similar misclassification rates, demonstrating that C-SFE does not compromise on effectiveness despite targeting fewer kernels.

2. Attack Efficiency: From line 460, we emphasized that C-SFE is more efficient in terms of hardware resource utilization and information transmission, making it better suited for integration with hardware Trojans.

W6. The Figure 2 needs to improve, the text are hard to read on part (b).

Thank you for your suggestion. Figure 2 has been revised for readability improvement.

W3. Some of assumptions are also not well established and needs clarified. For example line 265 "Avoid attacking the model’s first layer an the final classification layer, as attacks on these two layers can have a significant impact on the model and are therefore more likely to be closely scrutinized by security engineers." I doubt if that is the principle machine learning engineers follow; even if that is the case, how do they scrutinize these layers? There is no standard list of model parameters.

In our work, the reasons for us to avoid attacks on the model's first and final classification layers are based on the following considerations:

1. Importance of Key Layers: The first layer is usually responsible for extracting primary features from the input data, while the final classification layer directly affects the model's output results. Tampering with the parameters of these two layers can lead to a significant decline in model performance. For example, TBT [1] and SBA [2] exploit this characteristic. Additionally, the parameters identified by T-BFA also tend to be distributed in the final classification layer.

2. Priority in Security Detection: In actual security reviews and model validation processes, security engineers often prioritize parts that have a significant impact on overall performance. Due to the special status of the first and last layers, they are more likely to become focal points of inspection.

Regarding your questions about "how to scrutinize these layers" and "there is no standard list of model parameters," our understanding is:

1. Parameter Verification: Although there is no unified list of parameters, security engineers may perform checks on the model's key parameters (especially layers that have a significant impact) [3], [4].

2. Anomaly Behavior Monitoring: Security engineers may detect anomalies by monitoring the model's input and output behaviors. For example, if the output distribution of the model has significantly changed, they may trace back to inspect the output layer and its parameters.

[1] Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. Tbt: Targeted neural network attack with bit trojan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.13198–13207, 2020.

[2] Yannan Liu, Lingxiao Wei, Bo Luo, and Qiang Xu. Fault injection attack on deep neural network. In 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 131–138. IEEE, 2017.

[3] Qi Liu, Wujie Wen, and Yanzhi Wang. Concurrent weight encoding-based detection for bit-flip attack on neural network accelerators. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2020, 2020a.

[4] Jingtao Li, Adnan Siraj Rakin, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Radar: Run-time adversarial weight attack detection and accuracy recovery. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 790–795. IEEE, 2021.

Q4. The authors inject a Trojan into the EDA tool, potentially enabling triggers or malicious optimization on the hardware. I believe this point should be emphasized, with more detail on the significance and application value of such EDA tools. However, this brings up a broader issue: an EDA tool superior to others would likely dominate the market, making it challenging for a provider to practically act as an attacker? Could the authors discuss how realistic their threat model is?

We would like to clarify that the proposed HT is not embedded within any EDA tool itself, rather it is integrated into the Verilog files as indicated in lines 181 and 190 of the manuscript. Specifically, the HT is introduced in the source code and can be synthesized by EDA tools to generate the bitfile or layout, in which the EDA tool is merely a part of the VLSI design flow (as shown in Figure 1 of the manuscript), thus not compromising the EDA tool directly. As an example, the source code of HT can be found at line 1196 in LoopConv.scala and line 544 in Scratchpad.scala within the anonymous GitHub link provided in the manuscript. The Verilog file generated for synthesis with the EDA tool has also been uploaded to GitHub. Additionally, while a single EDA tool dominating the market might suggest a lower likelihood of it acting maliciously, security vulnerabilities remain an inherent risk. We believe dominance does not guarantee immunity from internal threats or supply chain compromises. For example, the security vulnerability in Intel HID Event Filter (CVE-2024-25561) [1] and Intel Agilex® FPGA Firmware Advisory (CVE-2024-25576) [2] demonstrate that even well-established tools can host vulnerabilities that allow escalated privileges or other security breaches.

It is believed that the proposed threat model is realistic and should not be omitted. As mentioned in the paragraph starting from line 114 of the manuscript, our threat model focuses on the growing field of automated AI accelerator generation platforms, however it is a domain where security has not been extensively explored. By abstracting critical security modules as shown in Figure 1, we aim to expose potential security risks and ensure that it has adequate security guarantees when commercialized in the future. In addition, our model has been validated across different hardware platforms and applies to any design framework adhering to the AI accelerator design paradigm, including but not limited to the Gemmini and GeneSys [3], [4] platforms. This widespread applicability underlines the practical relevance and urgency of addressing such threats.

[1] Intel. Intel Security Advisory SA-01089, 2024. URL https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-01089.html. Accessed on November 14, 2024.

[2] Intel. Intel Security Advisory SA-01087, 2024. URL https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-01087.html. Accessed on November 14, 2024.

[3] Soroush Ghodrati, Sean Kinzer, Hanyang Xu, Rohan Mahapatra, Yoonsung Kim, Byung Hoon Ahn,Dong Kai Wang, Lavanya Karthikeyan, Amir Yazdanbakhsh, Jongse Park, et al. Tandem processor: Grappling with emerging operators in neural networks. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, pp. 1165–1182, 2024.

[4] actlab genesys. Genesys, 2024. URL https://github.com/actlab-genesys/GeneSys. Accessed on November 14, 2024.

Q3. Could you provide component consumption under a fair comparison scheme, such as comparing a clean accelerator vs an HT-injected accelerator, both using Vivado's Area Optimization mode, Default mode, latency optimization, and so on? It appears that the LoopConv module was mapped to LUTs, yet Fig. 6 does not clearly show the specific size and location. Based on Table 2, using the default optimization requires approximately 1,000 additional LUTs. This amount seems quite large for HT injection.

Thank you for your insightful question.

The table below presents a fair comparison scheme, comparing the differences in resource consumption between clean designs and malicious designs under Default mode, Area Optimization mode, and Performance mode. The Variation represents the rate of change in resource usage between clean and malicious designs within the same mode.

We would like to clarify that, as described in the paragraph starting at line 435 of the manuscript, the original intention of additionally including the Area Optimization mode in Table 2 was to illustrate that users (software engineers) typically do not pay attention to the optimization options of the EDA tool. Therefore, this aspect could potentially be exploited by attackers to make the HT more stealthy.

LoopConv is used to parse Rocket Custom Coprocessor (RoCC) commands and convert them into a series of memory access and computation instructions (implementation details are provided in LoopConv.scala in the GitHub link in the manuscript). It is implemented as sequential logic based on a state machine. As part of the accelerator, LoopConv is located within the gray-white area in Figure 6. Since the main purpose of Figure 6 is to show the layout of various modules in the entire SoC (e.g., controller, peripherals, accelerator), we did not further subdivide the internal modules of the accelerator or label them with different colors. Instead, we highlighted the HT, as it is the most critical component, to emphasize its importance.

We acknowledge that our HT implementation consumes about 1,000 additional LUTs under default optimization, which is indeed significant. This is partly because we used Chisel to write the HT code. While Chisel offers high-level hardware description capabilities, it may not leverage certain low-level optimization techniques available in Verilog, leading to extra resource usage. To reduce LUT consumption, the attacker could write the HT code directly in Verilog. In our threat model, this is feasible since the attacker has deep hardware design knowledge and can employ low-level optimizations. Directly using Verilog allows for finer control over the circuit and more efficient resource utilization, significantly reducing the additional LUT overhead. Therefore, although the current HT implementation requires slightly more resource usage, it does not affect the practicality and effectiveness of the proposed threat model.

Thank you for your valuable suggestions! We have addressed the concerns you raised below.

Q1. Could the authors explain why they chose the U50 platform ? For IoT applications, it would be more appropriate to use platforms like the Ultra96 or KRIA KR260.

The selection of U50 was primarily driven by its ability to support a larger systolic array (32×32) Gemmini-generated accelerator, compared to the 16 × 16 array on the Xilinx Kintex-7 series (such as xc7k325t). This larger array significantly enhances inference speed, which is critical for efficiently testing the accuracy of quantized models during our experiments, as illustrated in Figure 1 of the manuscript. On the other hand, we also acknowledge that platforms like the Ultra96 or KRIA KR260, equipped with ARM Cortex-A53 processors, may be more typical for IoT applications due to their balance of computational power and energy efficiency. However, our threat model is designed to be hardware-agnostic and applicable to various configurations, including those with smaller computational resources like the ARM Cortex-M series.

The tests on U50 and the Xilinx Kintex-7 (xc7k325t) board were conducted to demonstrate the flexibility and adaptability of our attack framework rather than suggest a specific IoT implementation. The Vivado project files for the Kintex-7 are available via the anonymous GitHub link provided in the manuscript.

In summary, while the U50 platform was chosen for its specific technical advantages in our experimental setup, the broader applicability of our threat model to IoT contexts remains valid across both high performance and low-power devices, as we've also validated on FPGA platforms similar to those you mentioned.

Q2. For inserted Trojans, can they pass through functional verification steps ?

Our response is: Yes. The reasons are as follows:

1. The Trigger module in the HT ensures its stealthiness. When the Trigger is not activated, the HT does not introduce any functional anomalies to the design. Since the HT's trigger conditions are difficult to satisfy, the Trojan cannot be detected during the regular function verification process.

2. In the proposed HT design, the primary responsibility of the Payload component is to replace target model parameters stored in on-chip memory with malicious values. Notably, this component's design does not directly affect the circuit's output, which represents a significant difference from previous Payload designs. For example, in encryption chips, HTs are typically deployed along key-related paths and positioned near the output. This arrangement means that the Payload's behavior directly impacts the chip's output, enabling security personnel to develop detection strategies based on this relationship. However, the proposed Payload design carries out the attack by replacing internal model parameters without directly affecting the output. This makes it difficult to detect using methods based on test vectors [1], [2] and information-flow tracking [3].

3. We have experimentally confirmed that machine learning-based detection methods [4] are also unable to detect the proposed HT. Additionally, using Weight Clustering [5] does not mitigate the proposed attack.

[1] Chakraborty, Rajat Subhra, Francis Wolff, Somnath Paul, Christos Papachristou, and Swarup Bhunia. "MERO: A statistical approach for hardware Trojan detection." In International Workshop on Cryptographic Hardware and Embedded Systems, pp. 396-410. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009.

[2] Saha, Sayandeep, Rajat Subhra Chakraborty, Srinivasa Shashank Nuthakki, Anshul, and Debdeep Mukhopadhyay. "Improved test pattern generation for hardware trojan detection using genetic algorithm and boolean satisfiability." In Cryptographic Hardware and Embedded Systems--CHES 2015: 17th International Workshop, Saint-Malo, France, September 13-16, 2015, Proceedings 17, pp. 577-596. Springer Berlin Heidelberg, 2015.

[3] Wei Hu, Baolei Mao, Jason Oberg, and Ryan Kastner. Detecting hardware trojans with gate-level information-flow tracking. Computer, 49(8):44–52, 2016.

[4] Shih-Yuan Yu, Rozhin Yasaei, Qingrong Zhou, Tommy Nguyen, and Mohammad Abdullah Al Faruque. Hw2vec: A graph learning tool for automating hardware security. In 2021 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pp. 13–23. IEEE, 2021.

[5] Zhezhi He, Adnan Siraj Rakin, Jingtao Li, Chaitali Chakrabarti, and Deliang Fan. Defending and harnessing the bit-flip based adversarial weight attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14095–14103, 2020.

Dear Reviewer UkBM,

As the discussion period is nearing its conclusion, we kindly ask if you could review our response to ensure it addresses your concerns. Your feedback is greatly appreciated.

Thank you for your time!

Best,

Authors

W5. In some written sections, the use of the word "parameter" becomes a bit confusing whether it is the model parameters of DSE parameters (ex: line 190).

Thank you for your comment. We have clarified them in the manuscript.

W6. Minor issue: double quotations were not properly handled. (line: 394)

Thank you for your comment. We have revised them in the manuscript.

W7. Y axises of Figure 6 are not named.

Figure 6 presents the Vivado-generated layout of the accelerator with HT, which is primarily used to illustrate the distribution and connectivity of internal resources within the FPGA. As this layout corresponds to a specific FPGA board, it does not feature labeled x-axis and y-axis. The HBM label below the image simply indicates that the accelerator utilizes High Bandwidth Memory (HBM) as its off-chip memory.

W1. The evaluations of this threat model can be further extended to other platforms such as GeneSys.

Thank you for bringing up this important aspect.

As shown in line 206 of the manuscript, the proposed threat model can be applied not only to the Gemmini accelerator generation platform but also to GeneSys [1]. This is because GeneSys aligns with the AI accelerator design paradigm introduced in Figure 1 of the manuscript. Specifically:

1. "Cross-domain Applications" [2] corresponds to the user-input trained model (the blue section on the left in Figure 1).
2. "Architecture Stack" is used to determine the optimal hardware configuration (such as the number of processing elements, bit-widths, and on-chip buffer configurations) and corresponds to the "Exploration unit" in the Hardware build flow (the left yellow section in Figure 1).
3. "Compilation Stack" is used to convert the input model into Instruction Set Architecture (ISA). For the Tandem Processor, it generates Tandem ISA and Systolic Array ISA; for Gemmini, it generates RISC-V ISA and Rocket Custom Coprocessor (RoCC) commands. This part corresponds to the "Software build flow" (the right yellow section in Figure 1).
4. "FPGA/ASIC Implementation" corresponds to the "VLSI design flow" (the left yellow section in Figure 1).

Therefore, in the GeneSys-based automatic accelerator generation platform, the proposed threat model can still explore the model's vulnerabilities during the "Cross-domain Applications" and "Compilation Stack" stages, and insert malicious information into the Tandem Processor's Systolic Array ISA.

[1] Soroush Ghodrati, Sean Kinzer, Hanyang Xu, Rohan Mahapatra, Yoonsung Kim, Byung Hoon Ahn,Dong Kai Wang, Lavanya Karthikeyan, Amir Yazdanbakhsh, Jongse Park, et al. Tandem processor: Grappling with emerging operators in neural networks. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, pp. 1165–1182, 2024.

[2] actlab genesys. Genesys, 2024. URL https://github.com/actlab-genesys/GeneSys. Accessed on November 14, 2024.

W2.

It would have been better to have runtime performance numbers compared to the benign platform.

Thank you for your insightful feedback.

Our design ensures that HT does not affect inference time for the following reasons:

• HT Trigger in sequential logic: As depicted in Figure 2(b) and detailed in the LoopConv.scala script (line 1196) available in the supplementary GitHub repository, the HT trigger can be designed in a sequential logic, ensuring non-intrusive execution.
• Parallel execution: The Trigger and the Convolution configuration module in the accelerator execute in parallel, leading to overlapping execution times without incurring additional delays.
• Payload in combinational logic: The payload of the HT, implemented as a simple combinational logic circuit (refer to Scratchpad.scala, line 544), meets timing constraints without causing timing violations. Furthermore, empirical tests conducted under the hardware configuration described in our manuscript show that performing inference on a single image using ResNet-18 architecture requires approximately 4,287,063 clock cycles, which demonstrates no significant deviation from the expected performance norms. Therefore, the presence of HT does not adversely affect the accelerator's runtime performance.

Would there be a larger difference when the test dataset size increases?

Actually, our response is NO. There would not be a significant difference. Due to the deliberate design of the HT trigger, which is configured to activate very infrequently. Even as the dataset size increases, the probability of triggering the HT remains very low. This design ensures that the HT does not significantly impact the overall behavior of the system using different test dataset sizes, while maintaining a consistent and predictable performance.

W3. The discussion section (4.4) of the paper lacks depth. Perhaps to justify the findings of this section, it would be better to includethem in the appendix.

Thank you for your suggestion. We have moved it to the appendix.

W4. No clear numbers were presented how this would change for being and malicious platforms.

Thank you for highlighting this aspect.

The exploration algorithm is executed solely during the generation phase of the accelerator (see the yellow section in Figure 1) and is not involved during the runtime phase. Specifically, during the generation phase, line 408 in the manuscript indicates that the runtime of C-SFE is approximately 8 minutes. It is noteworthy that the total time from the user's model input to bitfile generation is around 50 minutes. Since C-SFE runs in parallel with the overall process, it does not affect the total time.

Thank you for your valuable suggestions! We have addressed the concerns you raised below.

Q1. Would a simple validation of model parameters at the end of execution, along with the classification results, show that the parameters have changed?

Thank you for your insightful question.

Our response is: No. The reasons are as follows:

1. The Trigger design in the HT ensures its stealth, as shown in line 197 of the manuscript. The random activation of the Trigger makes it difficult for security engineers to determine when to perform parameter validation.

2. When the Trigger is deactivated, the malicious parameters in the on-chip memory will be overwritten by clean model parameters from the off-chip memory at the next load, allowing the model to function normally.

Q2. Instead of carefully selecting which parameters to change, how would the results look like if parameters were changed randomly?

Thank you for bringing up this important aspect.

The table below compares the number of parameters altered by the proposed C-SFE versus random attack and their impact on accuracy across three models. For random attack, each parameter count was tested in 10 trials, and the averages are reported. Taking ResNet-18 as an example, reducing the model's Top-1 accuracy to 0.1% requires modifying over 100,000 parameters with a random attack, whereas C-SFE achieves the same result by modifying only 12 parameters. The code for the random attack has been uploaded to the anonymous GitHub link provided in the manuscript.

It is noteworthy that exploration algorithms suitable for HT need to consider not only whether the attack parameters can achieve the desired effect but also whether locating these parameters during the accelerator's runtime phase will consume excessive hardware resources. Therefore, even if a random attack is effective, the random positioning of these parameters may cause HT to expend a significant amount of hardware resources in locating them. Regarding existing adversarial weight attacks, many studies outperform our proposed C-SFE algorithm in terms of the number of bits attacked (e.g., T-BFA). However, current adversarial weight attack methods do not impose constraints on the regularity of the identified parameter locations, making them unsuitable for HT design. This is the motivation behind our emphasis in the paragraph starting at line 242 of the manuscript on the need to design an HT-friendly attack algorithm.

Q3. Why different numbers of kernels and parameters were selected for various models?

Thank you for your insightful question.

The selection of different numbers of kernels and parameters across various models is dictated by their varying capabilities in feature extraction. Our approach, C-SFE, adapts to these differences by selecting the set of target parameters for each model. It should also be noted that, in our threat model, the attacker's access is restricted to only 50 images used for quantization calibration. Our experimental results demonstrate that C-SFE can efficiently identify the minimal set of regular model parameters that classifies all 50 images into the target category. However, given the limited size of the available calibration dataset, which indicates the attacker does not have information about the full dataset, targeting additional parameters does not expand the attack’s impact beyond these 50 images.