We thank the reviewer for the valuable feedback and insightful questions. We appreciate the recognition of our contributions and the constructive suggestions, which have helped us further improve the quality and clarity of our work. Below, we address each concern in detail.

Q1: What is the performance difference between DDE and higher-order solvers like Runge-Kutta?

The DDE algorithm is currently based on discretizing nmODE with the Euler method. Technically, it also supports all higher-order solvers. We conducted further experiments on MobileODEV1 with other higher-order solvers on CIFAR-10, as shown in the table below:

While higher-order solvers like Runge-Kutta can theoretically improve accuracy, our preliminary experiments show that, in the context of lightweight discrete ODE models, their additional computational overhead does not translate to significant accuracy gains compared to DDE. It is very important for us to include these results in Table 7 in the revision.

Q2: Is it possible to apply COS modules to NLP or speech?

Excellent question! The COS module is a general mechanism designed for modeling interactions among feature channels. At its core, it functions similarly to an RNN block, where $y^l$ can be interpreted as the hidden state at time step $l$. This design makes COS inherently adaptable to non-vision domains such as NLP and speech.

For instance, in Transformer-based models, COS could potentially be leveraged to capture token-wise or feature-wise dependencies, enhancing the model’s ability to represent complex relationships within sequences. We also encourage exploring the use of COS modules in various sequence modeling tasks beyond vision, as their flexible structure may offer benefits across a wide range of applications.

Adapting COS to these domains would require redefining the dynamics over ODE or time steps, which we consider a promising direction for future work.

Q3: Why omit MobileNetV3/V4, MobileOne?

We define MobileNetV3/V4 and MobileOne as our baselines across three tasks. For the reason why ODEV variants are not provided, please refer to Q1 of Reviewer J9oG.

Q4: How does COS compare to dynamic convolution or attention-based channel mixers (e.g., in EfficientFormerV2)?

We compare COS to dynamic convolution and attention-based channel mixers in EfficientFormerV2 on CIFAR-10, as shown in the following table. DY-MobileNetV1 denotes replacing COS modules with dynamic convolutions, $K$ means convolution kernel number. SA-MobileNetV1 denotes replacing COS modules with the attention downsampling proposed in EfficientFormerV2 for mobile devices.

Q5: COS reduces parameters but increases FLOPs (Tab 7). Is this favorable on edge devices? Are there any energy consumption measurements on edge devices?

In Appendix 6.7, our practical deployment shows that this trade-off is favorable for edge devices. We have implemented our lightweight MobileODE-based breast screening system on a Lenovo Legion R9000P laptop (NVIDIA GeForce RTX 4060 GPU with 8GB VRAM, AMD Ryzen 9 7945HX CPU, 15.3GiB DDR5 RAM), running Ubuntu 20.04.4 LTS. During operation, GPU memory usage was 1708/8188 MiB and GPU utilization was only 19%, indicating low and stable resource consumption.

Although we have not directly measured energy consumption, the low hardware usage suggests that the increased FLOPs do not adversely affect deployment efficiency on edge platforms.

Q6: Why $\sqrt{C}$ factorization for $\Phi$? Ablate against linear (C×k) or low-rank (C×r×r) alternatives.

Thank you for your question regarding the use of $\sqrt{C} × \sqrt{C}$ reshaping in our design. We would like to clarify that our approach does not involve matrix factorization or low-rank decomposition of a parameterized transformation matrix $\Phi$. Specifically, we reshape the 1D channel feature vector of size $C$ into a 2D matrix of size $\sqrt{C} × \sqrt{C}$ solely for the channel-view of applying subsequent certain operations (such as batch normalization and bilinear interpolation in MobileODEv2).

Therefore, the comparison with linear (C×k) or low-rank (C×r×r) alternatives is not directly applicable. We will clarify this design choice and its motivation in the revised manuscript to avoid confusion.

Q7: Why not test on full ImageNet-1K? Accuracy on IN-tiny (64×64) may not reflect real-world performance.

Please refer to Q1 of Reviewer MTGp.

Concerns in the weakness.

W1: The bilinear interpolation used for channel reshaping may bottleneck scalability.

Thank you for highlighting this issue. We agree that as the expansion factor $e$ increases, bilinear interpolation may lead to information loss between channels and diminishing performance gains. The ablation study on $e$ in our work is intended to demonstrate that simply increasing $e$ for performance improvement should not be the main focus of MobileODEV2, which aligns with the reviewer's observation.

W2: COS updates channels sequentially (Algorithm 1), which limits parallelism; for long ODE trajectories (large L) this can increase inference latency.

The sequential channel update in COS is motivated by the explicit Euler discretization in ODE solvers, which is designed to capture dependencies across channels. While this sequential nature can limit full parallelism—potentially increasing inference latency for long ODE trajectories (large $L$)—our experiments show that the relationship between $L$ and latency is approximately linear, as demonstrated in Table Q2 for Reviewer MTGp. Thanks to the lightweight design of our architecture and the significant reduction in overall model parameters, the actual increase in inference time remains modest in our benchmarks. Moreover, we anticipate that future advances in hardware and software (e.g., pipelined or block-parallel COS updates) could further mitigate this limitation, offering even greater efficiency.

W3: The "extra-lightweight" claim is overstated: ShuffleNet V1 (1.1 M parameters) is still smaller than MobileODE-V1 (1.14 M) as shown in Table 3.

We acknowledge that, as shown in Table 3, ShuffleNet V1 (1.1M parameters) has a slightly lower parameter count than MobileODE-V1 (1.14M). However, our “extra-lightweight” claim is primarily based on the flexibility of the MobileODE architecture with respect to the $L$ setting. As demonstrated in the table below, MobileODE can be scaled down to just 0.885M parameters when $L=3$, while achieving performance closely matching that of ShuffleNet V1 (1.0×) on CIFAR-10.

Therefore, MobileODE offers not only competitive parameter efficiency but also scalable flexibility—allowing it to achieve even smaller sizes without significant performance loss. This adaptability makes MobileODE particularly suitable for resource-constrained scenarios and further supports our “extra-lightweight” claim.

We thank the reviewer for these insightful questions, which will help guide our future research directions. We also appreciate the reviewer’s comments on our writing, which have improved the clarity and rigor of our paper.

Q1: Why not provide ODEV variants for MobileNetV3 and V4?

MobileNetV3 and V4 are fundamentally built upon the backbone architectures of MobileNetV1 and V2. Their improvements mainly involve incremental tricks and optimizations, rather than substantial structural changes to the core architecture. This is why our comparisons focus on V1 and V2, as they represent the foundational designs of the MobileNet series.

Importantly, most of the tricks introduced in V3 and V4 can also be integrated into MobileODE. Exploring the incorporation of these enhancements, as well as further improving MobileODE with techniques such as NAS (Neural Architecture Search) and knowledge distillation, is an active area of our ongoing research.

Q2: Inconsistency between the paper and the provided source code.

We thank the reviewer for carefully pointing out the discrepancy between the manuscript and the provided code regarding the activation function applied after the increment operation. In fact, we have implemented two versions of the activation: one using ReLU6, as described in the paper, and another using torch.maximum(epsilon, x). We apologize for not clearly documenting this in the README file.

During our ongoing experiments with large step sizes $L$, we observed that enforcing non-negativity with torch.maximum(epsilon, x) helps prevent invalid step sizes and achieves comparable, and in some cases slightly better, numerical stability compared to ReLU6. We will update our findings in the new appendix of the manuscript and revise the code documentation to clarify this point and ensure consistency.

Concerns in the weakness.

W1: Hardware specifications for latency measurements.

All latency measurements were performed on a single NVIDIA RTX 4090 GPU. The reported latency represents the average inference time per batch, with a batch size of 16.

Additionally, we explore the impact of the hardware platform, as detailed in Table Q1 of Reviewer mANM. The results show that our model consistently outperforms MobileNetV1 in latency on both CPU and GPU. Furthermore, performance can be further optimized on TPU, while also revealing additional optimization potential in MobileODEs.

W2: Writing issues

In the revised manuscript, we have updated Figure 2 by introducing distinct colors for different $\Delta t$ values to improve clarity.

Additionally, we have corrected the axis in subfigure (c) of Figure 4 to range from 0 to 100% in the revised version.

Q1: On Latency Benchmark Details.

We appreciate the reviewer’s suggestion to provide a more detailed description of our latency benchmarks. As shown in the table, our model consistently outperforms MobileNetV1 in latency on both CPU and GPU. On the TPU, however, it is slightly slower, which we attribute to the sequential computation pattern of the COS module. We believe this overhead can be mitigated through hardware‑specific optimizations, especially as future accelerators become better optimized for directly differentiable models such as COS.

To address the reviewer’s concern, we will include additional details on the hardware configurations and the benchmarking protocol in the revised manuscript.

Q2: On Motivation and Ablation Studies.

Thank you for your insightful feedback. In response, we further investigate the latency of depthwise convolution (dw) and pointwise convolution (pw) on a single RTX 4090 GPU to provide a more detailed comparison. The following table, which aligns with Table 1 in our paper, clearly demonstrates that the original pointwise convolution consumes a significant portion of the computational resources, while our COS module accelerates this process.

Furthermore, we explore the impact of replacing pointwise convolution with COS-based blocks, particularly in the context of other channel computing modules and higher-order solvers. As detailed in Table Q1 and Q4 of Reviewer Tqqz, the results highlight how the COS module enhances both accuracy and computational efficiency.

First, we would like to thank the reviewer for carefully pointing out the writing issues. We have made the necessary corrections in the manuscript.

Q1: No Evaluation on Full ImageNet-1k.

Thank you for pointing out the lack of experiments on the full ImageNet-1k dataset. Due to computational resource constraints and the high cost of training ultra-lightweight models at scale, we focused our evaluation across diverse data domains on three popular tasks. In line with your concern about real scalability and robustness on high-resolution, real-world tasks, we conducted experiments on ImageNet-R (diverse textures and styles) and ImageNet-A (natural adversarial examples) at a resolution of 256 × 256, as detailed in Table 8 of the Appendix. Notably, MobileODEV1 and MobileODEV2 demonstrated substantial improvements over baseline models in the ImageNet-A experiments.

Nevertheless, we agree that a full ImageNet-1k evaluation would further strengthen our claims. In our study of MobileODEV1 on ImageNet-1k, we observed rapid learning progress in the initial stages, which subsequently decelerated as the model approached the performance of MobileNetV1. This phenomenon arose because the t value tended to approach zero in later iterations due to the ReLU6 activation function, thereby hindering learning performance. We hypothesize that large-scale data may excessively penalize the $t$ value, impeding the learning process. Based on these observed trends, we anticipate that MobileODE will maintain its strong efficiency-accuracy tradeoff on ImageNet-1k as well, particularly if an epoch-wise optimization strategy is adopted.

Q2: Provide a detailed analysis of how increasing the number of DDE steps (L) affects inference latency.

We appreciate the reviewer's concern regarding potential bottlenecks in real-time deployment as $L$ increases. To provide a clearer analysis, we evaluated latency at larger step intervals (every 10 steps), as shown in the table below. Latency was measured on a single NVIDIA 4090 GPU with a batch size of 16. The results indicate that inference latency increases approximately linearly with $L$, allowing for flexible adjustment in practical deployments. Notably, compared to other higher-order derivative methods listed in Table Q1 of Reviewer Tqqz, our DDE algorithm demonstrates favorable computational efficiency.

Q3: Consider evaluating on additional robustness benchmarks such as ImageNet-C (corruptions), ImageNet-A (natural adversarial examples).

We conducted experiments on ImageNet-A, as detailed in Table 8 of the Appendix. The results demonstrate that MobileODEs achieve substantial improvements over the baseline models.

Q4: Visualizations or analyses of how feature representations evolve across the $L$ steps.

We fully agree that such visualizations and analyses (e.g., activation similarity matrices, t-SNE/PCA of intermediate outputs) would provide important insights into how feature representations change over discrete layers and the roles of different steps.

However, according to this year's rebuttal guidelines, we are not permitted to include PDFs, links or visualizations directly in the response. Nevertheless, we recognize the value of such analyses and have included detailed feature visualizations and analyses in the Appendix of our revised manuscript. We hope these additions will help clarify how the internal representations and states evolve across the $L$ steps, and provide a deeper understanding of the COS module's dynamics.

Q5: Why not compare to existing ODE-inspired architectures like Neural ODEs, NODE-ResNets, or recent state-space models?

We did not directly compare with existing ODE-inspired architectures for the following main reasons. We also acknowledge that the higher-order derivative methods listed in Table Q1 of Reviewer Tqqz are more appropriate baselines in our context.

1. Different Objectives: Our work aims to develop ultra-lightweight, mobile-friendly models, whereas Neural ODEs, NODE-ResNets, and state-space models are typically designed for higher model capacity and are not optimized for extreme efficiency or deployment on resource-constrained devices.

2. Computational Cost: Traditional ODE-based networks often rely on the odeint package, which can incur significant computational overhead and potential precision loss. By discretizing the computation process, our approach not only reduces computational costs but also increases transparency and controllability compared to the black-box nature of standard ODE solvers.