Defending Deep Neural Networks against Backdoor Attacks via Module Switching

We thank the reviewer for their insightful feedback. Below, we address each point in detail.

---

W1: The theoretical analysis is based on a two-layer network without non-linearity; suggest including non-linearity analysis.

A1: Our theoretical analysis focuses on the linear activation scenario, and empirical results show that non-linear activations in Figure 5 exhibit the same trend: switched models consistently show greater backdoor deviation than WAG models.

We hypothesize this occurs because parameter updates ($\Delta$) are much smaller than original weights ($W$) in backdoor fine-tuning. For ReLU: $f(x) = W_2 \cdot \text{ReLU}(W_1 \cdot x) = W_2 \cdot (M \odot (W_1 \cdot x))$, where $M$ is a binary mask. When $|\Delta| \ll |W|$, the sign pattern remains largely unchanged, making local behavior effectively linear under a shared mask.

Empirically, across four backdoored RoBERTa-large tasks, we observe $|W|$ = 65.8 $\pm$ 27.2, $|\Delta|$ = 1.28 $\pm$ 0.40, and $|\Delta|/|W|$ = 2.11% $\pm$ 0.46% for FFN weights. This small update magnitude supports the applicability of our linear analysis to non-linear networks and explains the consistent empirical trends.

---

W2: The analysis assumes all backdoor models are fine-tuned from the same pretrained model — a strong assumption that may not hold in practice.

A2: We clarify that fine-tuning from a shared pre-trained model is a standard and widely adopted practice. This is supported by Hugging Face stats: over 4,500 models are fine-tuned from roberta-large, and 3,600+ from vit-base. The roberta-large alone was downloaded over 1.2 million times monthly.

Given the prevalence of this fine-tuning practice and the additional security benefits offered by model merging, we believe this setting is realistic and likely to become even more popular and widely adopted in future.

---

W3: Experiments use backdoored models with the same target class but different attacks. Suggest evaluating: (1) same attack with same/different target classes, and (2) different attacks with different target classes.

A3: We test both cases and find that our method works much better than WAG when the involved models have different target classes. Due to space constraints, the detailed results are provided in our responses A1 and A2 to reviewer utYj.

These results confirm that our method remains robust even when the backdoored models differ in both attack types and target labels. In the special case where both models are attacked using the same method with the same target label, the shared bias cannot be removed, but this is a limitation that also applies to WAG, as averaging or switching has no effect when both models encode the same bias.

We believe the increasing diversity of real-world datasets may help reduce such cases by lowering the likelihood of models sharing identical backdoor biases.

---

W4: The analysis and model-switching algorithm do not consider benign utility. Unclear why benign utility remains unaffected in experiments.

A4: We clarify benign utility preservation from three aspects: (1) theoretical and empirical analysis in our manuscript, (2) prior literature, and (3) additional experimental results on switching domain-relevant models.

First, as shown in Section 3, since $M_i$ and $M_j$ share the same pretraining, their semantic knowledge is preserved by both weight averaging and our switching method, thereby maintaining benign utility. This is evidenced in Figures 2 and 5, where the WAG and Switched models remain close to the benign model in semantic space. As noted in W1, fine-tuning introduces only minor changes to pretrained weights.

Second, prior studies [2, 60, 63] show that merging models—either through averaging or selective strategies—can preserve task performance. DARE [63] further shows that even after dropping parameters from cross-task but homologous models, the merged model performs well on both tasks (Figure 2).

Third, we empirically validate this by merging models trained on stylistically different datasets (IMDB and SST-2). Our method maintains high benign accuracy (CACC) while greatly lowering ASR:

These results confirm that our method not only reduces backdoor vulnerabilities but also preserves benign task performance.

---

W5: The overhead is very high. It takes 6 hours to find a switching strategy.

A5: We clarify that the search needs to be performed only once per architecture, as the resulting strategy is transferable across models, modalities, and tasks. End users can directly apply the released strategy without incurring this cost.

Importantly, the search is conducted entirely on CPUs and does not require high-performance hardware such as GPUs. We believe this makes the method accessible and practical. We acknowledge exploring more efficient search strategies as valuable future work.

---

W6: It's unclear whether the model-switch method can be applied to other architectures beyond transformers.

A6: We use Transformers as our testbed due to their relevance to our primary baselines and strong performance across both text and vision tasks [A]. While we have not evaluated CNNs or emerging architectures, we acknowledge this limitation in our submission and consider it a valuable direction for future work.

As discussed in Section 4.2, we hypothesize that backdoors propagate through both feed-forward and residual connection paths—common components of modern neural networks (NNs). Consider a general NN structure:

$y = F_{W_n}(\cdots F_{W_2}(F_{W_1}(x_0) + x_0) + x_1 \cdots ) + x_{n-1}, \quad x_i = F_{W_i}(x_{i-1}) + x_{i-1}$

The key idea is to prevent adjacent weight modules from originating from the same model. In CNNs or MLPs, this could involve layer-level disruption or finer-grained kernel-level decomposition—directions for future exploration.

---

W7: Lacks evaluation of other attacks and defenses, such as Combat, UNIT, and others.

A7: Thank you for this suggestion. Our experimental design, which includes nine distinct experiments, focuses on validating our method's performance on Transformer architectures across both text and vision domains, building directly on the WAG baseline.

Regarding the recommended methods, while most focus solely on CNNs, we did investigate the two relevant Transformer-based methods:

• UNIT: Operates under a different assumption, requiring a large clean dataset (e.g., 5% of training data, i.e., 2,500 samples for CIFAR-10), which contrasts with our low-data setting of 20–50 samples per class.
• Combat: Our evaluation of the released code for ViT-Base on CIFAR-10 resulted in low clean accuracy (70%), a trend consistent with the original paper (Table 3). Further adaptation would be needed for a fair comparison.

We believe our current experiments provide a focused and comprehensive validation. We will add more detailed related work discussion in the revision.

---

W8: The empirical study at Line 137 doesn't include any training and leverages random noise.

A8: We clarify that our study involves proper training in two phases:

• Pretraining: The network is trained on 250K synthetic samples $(X, S \times X)$ using MSE loss, where $X$ is random input and $S$ the weight matrix.
• Backdoor fine-tuning: The model is then fine-tuned on 1,000 backdoor samples with outputs $(S + B) \times X$, where $B$ encodes the backdoor pattern.

We repeat this process 1,000 times to generate distinct networks, enabling robust analysis. Also, we test four activation functions and observe consistent results, supporting its generality.

---

W9: Why is one switching strategy sufficient for different attacks? Consider adaptive attacks where attacker knows strategy.

A9: Our method does not rely on identifying attack-specific shortcut patterns. Instead, it recombines modules from different sources to cover as many shortcut paths as possible, disrupting backdoor behavior across diverse attacks.

As a result, multiple switching strategies are effective—our method provides a systematic way to discover them. Figures 6 and 7 show two such strategies (one strong, one suboptimal), and additional runs with different seeds yield similarly effective variants.

This diversity offers robustness against adaptive attacks. Even if the attacker knows one strategy (e.g., by freezing unused modules), the defender can apply another to break the transmission path.

To validate this, we simulate an adaptive attacker aware of Strategy 1 (S1) and defend using Strategies 2 (S2) and 3 (S3). Results are shown below:

These results show that strategy diversity enables our method to remain effective under adaptive attacks.

---

W10: The NLP tasks are simple with 2-4 classes.

A10: We clarify that having 2–4 classes is typical for many NLP classification tasks and is a standard setting in the backdoor attack and defense literature. In fact, all 5 of the most recent works on text backdoor defenses ([2], [3], [68], [A], [B], cited in A2 to reviewer DvkL) adopt similar class settings. We follow these setups to enable fair comparisons.

Moreover, the datasets used in our study—SST-2, MNLI, and AG News—are widely recognized benchmarks with strong community adoption, with high citation counts (10K, 5K, and 8K). Their use ensures relevance and rigor in evaluating backdoor defenses.

Thank you for these insightful follow-up comments. We appreciate your continued engagement and the opportunity to further clarify our assumptions and address the remaining concerns.

---

A2. While in our initial rebuttal we listed roberta-large and vit-base as representative examples to show that practitioners often fine-tune models from the same pretrained backbone, we would like to clarify that this practice generalizes to many other models. For instance, the Hugging Face Hub includes over 14,000 models fine-tuned from bert-base, over 14,000 from distilbert-base, and over 3,600 from deberta-v3, with similar trends observed for many other foundation models. More importantly, as a proactive defender, the user has the flexibility to choose the defense method and to select which models to use, making this assumption both realistic and actionable.

---

A4. Regarding the reviewer’s suggested case of adversarial manipulations such as layer shuffling: while it is possible in practice, to our knowledge, such operations are rarely discussed or evaluated in prior work.

To further explore this non-trivial setting, we tested random layer shuffling on roberta-large, shuffling 2/4/6/12/24 layers. The clean accuracy dropped from 96.3% to 56.4%, 59.7%, 52.4%, 49.1%, and 47.2%, which are close to random guess performance. These results show that such manipulations severely degrade clean performance—contrary to the goal of stealthy backdoor attacks, which aim to preserve utility and avoid detection.

We welcome the reviewer’s insights on how adversarial attackers might design more adaptive strategies that maintain clean accuracy while diverging from standard training behavior, and we are happy to further investigate such cases.

---

A5. Our method can run on both CPU and GPU, but there is no significant difference in runtime for strategy search, as it does not involve loading models or performing heavy computations such as matrix multiplications. Considering the cost and accessibility, CPU is the more preferable setting. Prior search-based model combination methods (e.g., DARE [63] and DAM [60]) typically require two steps: (1) loading models and (2) verifying their performance on validation sets—resulting in high computational overhead that must be incurred for every execution. In contrast, our method is independent of specific model parameters and only needs to be executed once per architecture, making it more practical for scalable deployment.

---

A6. Thank you for the comment. We would like to clarify that our work has explored various architectures, ranging from synthetic two-layer networks with varying activation functions to both text and vision Transformers. These represent both simple and advanced neural architectures, showing the generalization of our method and offering an interpretable perspective.

We thank the reviewer for their insightful feedback. Below, we address each point in detail.

---

W1: The method assumes a two-stage backdoor training process, where models are first pre-trained on clean data and then poisoned. However, real-world attacks may train from scratch without a benign phase. Can the authors clarify how their method performs under such circumstances and provide experimental results under this setting?

A1: We appreciate this insightful question. To our knowledge, injecting backdoors during the fine-tuning stage is the more common practice, as pretrained models are typically released by reputable organizations and are generally assumed to be clean. This assumption is also adopted in most existing backdoor attack studies and is commonly used in defense research as well.

While we acknowledge that training from scratch with backdoor injection is theoretically possible, we are not aware of prior work adopting this setting for large pretrained models, likely due to the substantial computational cost. Moreover, we were unable to fully simulate this setting due to resource limitations. To approximate it, we consider a two-stage training process inspired by [22] : (1) we first fine-tuned a pretrained model on a backdoored IMDB dataset; (2) we then further fine-tuned this model on clean SST-2 data. We applied both WAG and our method to defend against such a model. The results are shown below:

We observe that both WAG and our method achieve strong defense performance. We attribute this to the second clean fine-tuning stage weakening the backdoor effect, making it easier to defend. These results provide insight into our method's robustness in such a simulated setting.

[22] Kurita et al. Weight poisoning attacks on pre-trained models. (ACL 2020)

---

W2: The presentation of Figure 3 is somewhat confusing and needs to be revised. Appropriate explanatory text should also be added to clarify its content.

A2: Thank you for this helpful feedback. Figure 3 is intended to provide an intuitive illustration of a possible switched combination between two models, based on the rules introduced just above the figure. The red and blue nodes represent weight modules in the merged model, indicating their source. The origin of each module is determined by both intra-layer and cross-layer connections. The arrows depict the feedforward paths followed during model inference.

To improve the clarity of the figure, we plan to revise both its visual structure and accompanying explanation. Specifically, we plan to:

• (1) sketch the structure of a Transformer model, including the self-attention and feedforward (FFN) blocks and their associated weight parameters;
• (2) clarify the notation used for the six modules in the current figure—for example, Q and K refer to $W_{\text{query}}$ and $W_{\text{key}}$ from the self-attention block, while I and P correspond to the input and output weights of the FFN (MLP) block;
• (3) annotate examples of each switching rule directly in the figure, such as adding explanatory text for colored Q and K nodes to illustrate the Type 1 intra-layer adjacency penalty.

We would also be grateful for any additional suggestions the reviewer may have to further improve the figure's clarity and presentation.

---

W3: The WAG-based method relies on access to at least two models (benign or poisoned), which is a stronger assumption than having a small clean dataset. It significantly limits the feasibility of applying such methods in real-world scenarios.

A3: We would like to clarify the practicality and feasibility of our method from four aspects: (1) Standard practice in model merging research, (2) Domain-related merging is feasible, (3) Real-world model availability, and (4) No assumption of clean models.

First, our method follows a widely adopted setting in prior model merging work, such as WAG [2] and DAM [60], where models with the same architecture are assumed to be available to enable training-free knowledge transfer [19,33] or for use in multiple expert-merging scenarios [A,B]. We will add additional references to the model merging literature in the related work section.

Second, to enhance practicality, the WAG paper (Table 4) also demonstrates that models trained on related domains can be effectively merged. For example, IMDB, Yelp, Amazon reviews, and SST-2—all involving movie or product reviews—can be combined despite stylistic differences. We evaluate our method under a similar domain-relevant setting by merging two poisoned models (IMDB-Badnet and SST-2 InsertSent), and observe strong defense performance:

Third, structurally compatible models are often accessible in practice. For instance, Hugging Face hosts over 100 roberta-large models in the sentiment domain, including 82 tagged as sentiment, 28 as sst2, and 13 as imdb, making model compatibility feasible in real-world scenarios.

Finally, our method does not assume that either model is benign. It remains effective even in a challenging yet more practical setting where both source models may be compromised.

[A] He et al. Merging Experts into One: Improving Computational Efficiency of Mixture of Experts. (EMNLP 2023)
[B] Tang et al. Merging Multi-Task Models via Weight-Ensembling Mixture of Experts. (ICML 2024)

---

W4: Although the search is performed once, it takes around six hours—exceeding the time required to train a model. With limited computational resources, such overhead is significant and cannot be overlooked. Suggest optimizing the search procedure to enhance its practicality.

A4: We would like to highlight that the evolutionary search is a one-time, offline cost per architecture, not per task or deployment. For example, the unified strategy found (Fig. 6) generalizes well across different model types (e.g., RoBERTa, BERT, DeBERTa, ViT), multiple tasks (SST-2, MNLI, AG News), and varying poisoning rates (20%, 10%, 1%) and attack methods. We believe that even for a new architecture, several hours of offline computation is a reasonable cost for practical deployment. We also plan to release the strategies discovered as public artifacts to support ongoing research on backdoor defense.

Moreover, the search runs entirely on CPU and does not require high-end GPUs, making it accessible to users who cannot afford finetuning but still aim to perform secure inference. In scenarios where the data is poisoned, training a new model alone may not suffice, and identifying or removing the malicious data can be an even more computationally expensive process.

We acknowledge that the search procedure could be further optimized and view this as a valuable direction for future work.

---

W5: Table 3 shows that the performance of WAG and the proposed method is nearly identical when fusing three models. Would the gap shrink further with more models? Does this suggest that module switching offers no fundamental advantage over weight aggregation?

A5: Thank you for this observation. It is a well-documented phenomenon in both WAG [2] and DAM [60] that as the number of accessible models increases, the defense performance of model merging methods improves.

However, a common constraint in prior methods is the reliance on 3–6 models to achieve strong defense performance. When only two models are available, the attack success rate (ASR) often remains high. This trend is evident in Tables 2 and 5 of WAG [2], and Tables 2 and 3 of DAM [60], where performance improves significantly with 4–6 models but remains weaker with only two.

In addition, some methods designed for the two-model case—such as [29]—require one of the models to be clean. Otherwise, new vulnerabilities may arise, as discussed in [C] (Table 4).

Our work is motivated by this gap. We aim to provide an effective defense using only two models, without assuming either is benign. This makes our method more practical and deployable for end users, especially in constrained settings.

[C] He et al. TUBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning. (ACL 2025)

We thank the reviewer for their insightful feedback. Below, we address each point in detail.

---

W1: The pseudocode algorithms are formatted poorly and are not easy to read/follow. Please consider cleaning up the pseudocode.

A1: Thank you for this feedback. We provide clarification on the purpose and logic of the two algorithms below.

Algorithm 1: This algorithm assigns a model index (e.g., 1 or 2) to each module position in a transformer model. For two RoBERTa-large models (24 layers × 6 modules), this results in 144 positions.

• Lines 4–8: Randomly initialize a population of candidates (e.g., 100), each with a full index assignment. Evaluate each using heuristic fitness.

• Lines 9–19: Run evolutionary search for a fixed number of generations (e.g., 2 million): (1) select parents via tournament selection [41], (2) generate children through random mutation, and (3) retain the top-performing candidates (lines 16–17).

Algorithm 2: This algorithm identifies the cleaner model by comparing feature distances to a WAG-optimized dummy input for a suspect class.

• Lines 5–14: For each candidate class: line 7 optimizes a dummy input so the model predicts it as the target class; line 8 extracts its hidden representation; the mean cosine distance to clean samples from other classes is then computed. The class with the highest average distance is selected (line 14).

• Lines 15–20: Compute each switched model's feature distance to the WAG dummy input for the suspect class to select the cleaner model.

We would also appreciate it if the reviewer could highlight any specific lines that were unclear, as this would help us improve the presentation further.

---

W2: The experiment settings seem a bit outdated. Can the authors please provide justifications for the experimental selections, including model selection and other backdoor attacks examined/compared?

A2: We chose the models and backdoor attacks primarily based on the experimental setup of the baseline method WAG [2], to ensure a fair and controlled comparison. While WAG focuses on the text domain, we extend the evaluation to the vision domain to assess the broader robustness and generalizability of our method.

To validate the relevance of our selections, we conducted an up-to-date survey of recent top-tier venues. We found that both the models and backdoor attacks used in our study remain commonly adopted in the latest literature, supporting the rationale behind our design.

In the text domain, 4 out of 5 recent papers [2], [3], [68], [A], [B] adopt the same model family as ours, and all 5 study the same backdoor attacks—BadNet, InsertSent, and HiddenKiller—which we also include in our experiments.

For the vision domain, we selected the ViT model as it belongs to the same architectural family as our text encoder, allowing us to test the transferability of our strategy across modalities. Recent studies [60] and [F] also evaluate backdoor attacks on Vision Transformer models. The attacks we study, such as BadNet and WaNet, are widely used in recent literature [C]–[F], reaffirming their relevance.

[2] Arora et al. Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge. (ACL 2024)
[3] Yi et al. BadActs: A Universal Backdoor Defense in the Activation Space. (ACL 2024)
[60] Yang et al. Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace. (ICLR 2025)
[68] Zhao et al. Defense Against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization. (ICML 2024)
[A] Chen et al. PKAD: Pretrained Knowledge Is All You Need to Detect and Mitigate Textual Backdoor Attacks. (EMNLP 2024)
[B] Zhang et al. BadWindtunnel: Defending Backdoor in High-Noise Simulated Training with Confidence Variance. (ACL 2025)
[C] Xu et al. BAN: Detecting Backdoors Activated by Adversarial Neuron Noise. (NeurIPS 2024)
[D] Zhu et al. Breaking the False Sense of Security in Backdoor Defense Through Re-Activation Attack. (NeurIPS 2024)
[E] Hu et al. BBCaL: Black-box Backdoor Detection Under the Causality Lens. (ICLR 2025)
[F] Hu et al. A Closer Look at Backdoor Attacks on CLIP. (ICML 2025)

---

W3: Experiments can be enriched. For example, is there a reason why the authors only choose one model for the vision experiments while having three for the text experiments?

A3: We would like to clarify that our submission already includes 9 experiments—5 main experiments and 4 ablation studies—carefully designed to be comprehensive and support different aspects of our claims.

In the text domain, we use RoBERTa-large as the primary model and include two additional models in the ablation studies to demonstrate that our method generalizes across architectures. The vision experiment focuses on generalize the finding to another modality.

To further enhance the experiment on vision models, during the rebuttal period, we evaluated ViT-large (24 layers) using the same strategy as in Figure 6. The results are shown below:

The results are consistent to the claim for the advantage of our methods.

We hope this clarifies our experimental choices and demonstrates the broad applicability and robustness of our method.

---

Q4: The method is only evaluated on transformer-based models (BERT, RoBERTa, ViT). Can the authors please provide some insights on the proposed method for other model architectures, such as CNNs or MLPs?

A4: We focus on transformer models for two reasons: (1) our primary baseline method demonstrates backdoor defense capability in the text domain using a transformer model, and we retain the same setting to validate our claims; (2) we further demonstrate that our method has cross-modality effectiveness, and transformers are the architecture showing strong performance across modalities ([G], [H]), making them suitable for this evaluation.

While we acknowledge the lack of evaluation on CNNs and emerging architectures, we have explicitly mentioned this point in the Limitations section of our submission and regard it as a valuable future research direction. We are willing to share some insights on this matter.

As discussed in Section 4.2 of our submission, we consider backdoors to be mainly transmitted through both feed-forward and residual connection paths, which are essential components of most modern neural networks (NNs). Consider a general NN expression:

$y = F_{W_n}(\cdots F_{W_2}(F_{W_1}(x_0) + x_0) + x_1 \cdots ) + x_{n-1}, \quad x_i = F_{W_i}(x_{i-1}) + x_{i-1}$

We consider the key insight to be preventing adjacent weight modules from originating in the same model. In the context of CNNs and MLPs, we believe it is worth exploring either layer-level disruption or even finer-grained kernel-level decomposition, which we leave for future investigation.

[G] Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. (ICLR 2021)
[H] Khan et al. Transformers in vision: A survey. (ACM Computing Surveys 2022)

I want to thank the authors for the response. The authors partially addressed my concerns (e.g., W1 and Q4). However, for W2 and W3, I still believe that the authors should consider, especially for the evaluation on the vision tasks, including more recent attacks (e.g., BATT[1], AdaptivePatch[2]) in the later version. Such a practice will strengthen the overall quality of the paper.

[1]. BATT: BACKDOORATTACKWITHTRANSFORMATION-BASEDTRIGGERS -ICASSP 2023

[2]. REVISITING THE ASSUMPTION OF LATENT SEPARABILITY FOR BACKDOOR DEFENSES - ICLR 2023

We thank the reviewer for their insightful feedback. Below, we address each point in detail.

---

W1 & Q1: The suspect-class detection mechanism in Section 4.4 assumes that backdoored models share the same target class—an assumption that may not hold in real-world scenarios. Can the proposed method still work effectively when the backdoored models intended for switching have different target classes? Could the authors provide additional experimental results for this more realistic setting?

A1: Yes, our method remains effective in target-label-inconsistent scenarios, where the involved models have different target classes. We conducted additional experiments for both text and vision domains, where our method remains competitive with or outperforms the WAG baseline, as shown below:

These results confirm that our method is robust even when the backdoored models differ in their target labels.

---

W2 & Q2: The experiments do not evaluate cases where models are trained using the same backdoor attack. Does the proposed method remain effective when the backdoored models used for switching are generated with the same attack type?

A2: Yes, our method works much better than WAG, even when switching between models attacked by same method with non-identical labels. The result are shown below:

For cases that both switched model attack by same method with identical label, the significant bias cannot be fixed. However, the issues was also a limitation for WAG, as averaging or switching yields no change when models encode same bias. Analogously, exchanging ideas between individuals who share the same misconceptions may not lead to correction, but rather reinforce the existing viewpoint.

We believe that the increasing diversity in datasets may help reduce such cases, as it lowers the likelihood of models sharing identical biases.

---

W3 & Q3: The method described in Section 4.4 appears computationally intensive, yet the paper lacks adequate analysis of the computational overhead. Could the authors provide a thorough cost analysis for the entire defense mechanism?

A3: We provide a breakdown of the computational cost for the two main components of our method:

Switched Model Selection (Section 4.4): This step is computationally efficient due to the small number of data points involved. Specifically, Lines 7 and 8 of Algorithm 2 operate on a single dummy input, and Line 9 processes only 20–50 clean data points. We measured the total runtime on SST-2 and CIFAR-10 using a single NVIDIA RTX 4090 GPU:

• SST-2: 3.83 seconds total, with Lines 5–13 taking 2.3 seconds
• CIFAR-10: 43.08 seconds total, with Lines 5–13 taking 20.63 seconds

Although runtime may vary slightly with the number of classes, it remains under minute level which is a minor computational burden.

Evolutionary Strategy Search (Section 4.3): As reported in Section 5.1, the one-time search process takes approximately 6 hours for a 24-layer model. However, the search only needs to be performed once per architecture, as the resulting strategy is transferable across models (Table 16) and datasets (Table 7-9). Thus, end users can directly use the released strategy without incurring this cost.

Importantly, the search runs entirely on CPUs without requiring high-performance hardware such as GPUs, which makes the design accessible and computationally feasible. We welcome any suggestions from the reviewer, which are greatly appreciated and helpful in improving our work.