UltraLightUNet: Rethinking U-shaped Network with Multi-kernel Lightweight Convolutions for Medical Image Segmentation

We thank the reviewer for recognizing the effectiveness of UltraLightUNet's multi-kernel structures in capturing multi-scale contexts and achieving superior segmentation accuracy with minimal complexity. We also appreciate the constructive feedback and will address each of the reviewer’s comments in the following responses.

Q1. Although well-organized, the manuscript could benefit from a deeper focus on explaining how each module reduces computational costs while maintaining high performance. The method section lacks clear evidence or mathematical proof to support the model’s design, which may present a scientific limitation. Can you clarify the effects of the proposed modules in the model with clear evidence or mathematical insight? The current equations in the manuscript seem somewhat redundant, serving primarily as mathematical restatements. It would be helpful to provide more rigorous insights or proofs demonstrating how each module contributes to performance improvement and computational efficiency.

We appreciate the reviewer’s comment and provide additional clarification regarding UltraLightUNet’s design, which is grounded in clear theoretical concepts and validated through empirical studies and visualizations. Specifically:

Theoretical Basis: Most existing architectures in computer vision rely on foundational concepts rather than explicit mathematical proofs to justify their design choices. For example, Vision Transformers leverage self-attention mechanisms, and ConvNeXt uses large-kernel convolutions to enhance feature extraction. Similarly, our approach employs the multi-kernel trick to balance local and global feature extraction via multi-kernel depth-wise convolutions (MKDC) and incorporates depth-wise convolutions for lightweight computation. These foundational design choices, while not purely theoretical (or have no mathematical proof), ensure that UltraLightUNet achieves both high segmentation performance and extreme efficiency.

Empirical Validation: Ablation studies in Tables 4 and 5 of our initial submission quantify the individual and combined contributions of the modules. For instance, MKIR and MKIRA together significantly boost DICE scores, such as from 72.41% to 76.61% on the BUSI dataset, while the integration of all modules (MKIR, MKIRA, and GAG) achieves the best performance at 78.04%. This demonstrates how the design enhances segmentation accuracy while maintaining computational efficiency. We have additional abaltion experiments reported in Tables 6, 7, and 8 (in the Appendix of initial submission) which show the impact of our individual module over existing counterparts.

Visual Evidence: Activation heatmap visualizations (Fig. 4 and Section A.7 in our revised draft) further validate the practical impact of our modules. These visualizations show that MKIR, combined with CMFA, effectively attends to and refines critical regions in an image, such as lesion boundaries, improving segmentation quality.

To address the reviewer’s concern, we will revise the manuscript to contextualize our approach within the broader landscape of existing architectures that rely on foundational concepts rather than mathematical proofs.

Thank you for your effort and response. While some of my concerns (e.g., ablation results) have been addressed, I believe more focus is needed on providing theoretical support for the proposed model. The Learning Society should prioritize demonstrating technical insights and theoretical rationale over merely emphasizing performance improvements and simple interpretations. In current state, my score remains unchanged.

We sincerely thank the reviewer for the constructive feedback, and highlighting both the strengths and areas for improvement in our work. We deeply appreciate the recognition of UltraLightUNet’s originality and its contributions to computational efficiency and high segmentation quality, achieved through its novel lightweight architecture and innovative modules. Below, we address each concern raised by the reviewer and outline how we plan to incorporate these suggestions in our revised manuscript.

Q1. First, while the model is evaluated across various medical imaging datasets, these datasets are relatively straightforward, covering simple segmentation tasks and organs rather than more complex applications like CT/MRI tumor and lesion segmentation. How it would work on more complicated problems (e.g., 3D tumors/lesions) and multi-class segmentation problems?

We appreciate the reviewer’s suggestion to evaluate UltraLightUNet on more complex tasks like CT/MRI tumor and lesion segmentation. To address this, we conducted additional experiments on the MSD Task01_BrainTumour (multi-level MRI segmentation) and Task06_Lung (binary CT segmentation) datasets, and the results are presented in Table R5 below. We note that our UltraLightUNet3D-M achieves the best DICE scores in Tumor core (83.41%), Whole tumor (91.51%), and Lung cancer (71.53%) segmentation while maintaining remarkably lower computational costs compared to heavyweight methods like SwinUNETR (62.19M parameters, 328.6G FLOPs) and 3D UX-Net (53.01M parameters, 632.0G FLOPs). Furthermore, UltraLightUNet3D-M achieves the second-best DICE scores in Non-Enhancing Tumor and average Brain Tumor segmentation (78.95%), demonstrating its balanced performance across tumor subregions.

Compared to the existing lightweight method SlimUNETR, UltraLightUNet3D-M achieves better segmentation results on all tasks while maintaining similar computational efficiency (1.42M parameters, 7.33 GFLOPs vs. SlimUNETR's 1.78M parameters, 5.25 GFLOPs).

Additionally, our base model, UltraLightUNet3D, demonstrates competitive performance compared to heavyweight models like 3D UX-Net and SwinUNETR, while significantly outperforming SlimUNETR, achieving the lowest computational cost (0.453M parameters, 3.68 GFLOPs). These results validate UltraLightUNet’s ability to generalize to complex segmentation tasks with an excellent balance of accuracy and efficiency.

Table R5: Experimental results (DICE %) of 3D Brain tumor and Lung cancer segmentation on MSD Task01_BrainTumour (4-channel inputs) and MSD Task06_Lung datasets. #FLOPs are reported for 4-channel inputs with 96x96x96 volumes. Note: Tumor Core (TC), Whole Tumor (WT), Non-enhancing Tumor (NET).

Q4.1. The theoretical development is not solid. Authors reviewed Vision Transformers in the related work, but it is not related to the work in this manuscript. The network in the manuscript employs a convolutional neural network architecture and attention mechanisms.

We thank the reviewer for their thoughtful feedback and for pointing out areas for clarification regarding the theoretical development and relevance of Vision Transformers in our work. Below, we address this comment:

We included a discussion of Vision Transformers in the Related Work section to provide context on computationally expensive approaches, such as TransUNet and SwinUNet, which are widely used and popular in medical image segmentation. These methods demonstrate strong performance, but come with high computational demands due to their reliance on transformer-based architectures.

Our proposed multi-kernel design with depth-wise convolutions directly addresses these limitations by ensuring lightweight efficiency while maintaining high performance. This aligns with the motivation of our work, which emphasizes computational efficiency for resource-constrained environments. The inclusion of Vision Transformers in the Related Work highlights this contrast and establishes the relevance of our lightweight design in comparison to transformer-based approaches.

Q4.2. Additionally, insufficient theoretical development in the Method section, and it is unclear how this design improved the segmentation performance.

UltraLightUNet’s design is grounded in clear theoretical concepts, which are explained in the Method section and validated in the Ablation Study section:

• Theoretical Basis: Most existing architectures in computer vision rely on theoretical concepts to justify their design choices (e.g., Vision Transformers focus on self-attention). Similarly, our approach employs the multi-kernel trick to improve segmentation performance and depth-wise convolutions for lightweight computation. While our contribution is not theoretical in nature, these foundational concepts ensure that UltraLightUNet achieves both high performance and extreme efficiency.

• Method Section: We elaborated on how Multi-Kernel Inverted Residual (MKIR) and Multi-Kernel Inverted Residual Attention (MKIRA) blocks work together to balance performance and efficiency. The use of multi-kernel depth-wise convolutions (MKDC) enables adaptable feature extraction, while Convolutional Multi-Focal Attention (CMFA) enhances critical features.

• Empirical Validation: The Ablation Study (Tables 4 and 5 in our initial submission) demonstrates the impact of each module on segmentation performance. These results show how the theoretical design improves segmentation accuracy while maintaining computational efficiency.

To address this reviewer’s concern, we plan to further emphasize the connection between the theoretical design and its performance improvements in the revised manuscript, explicitly linking the multi-kernel design to segmentation accuracy.

Q3.1. The motivation is unclear. The author mentioned that several 3D segmentation networks, including 3D U-Net, SwinUNETR, 3D UX-Net, UNETR, nnU-net and nnFormer, have high computational demands, so they proposed their lightweight network. However, these baseline networks are proposed in 2021 and 2022, and in recent years many lightweight 3D segmentation networks have been proposed and this challenge has been tackled, such as [6][7][8][9][10]. However, authors didn't discuss and explore these lightweight networks.

Response:

We thank the reviewer for their thoughtful feedback regarding the clarity of the motivation and exploration of lightweight segmentation networks. Below, we provide clarifications and address the concerns raised:

Motivation: The primary motivation of our work is to address the computational demands of existing 2D segmentation networks by proposing an ultra-lightweight architecture that achieves competitive performance with significantly reduced parameters and FLOPs. Recognizing the versatility of our architecture, we then extend it to 3D by incorporating volumetric modules, ensuring that the same architecture works efficiently for both 2D and 3D segmentation tasks. This unified, resource-efficient approach allows UltraLightUNet to cater to a broader range of applications, including 3D volumetric tasks, with extreme efficiency.

Addressing Missing Lightweight Networks: The reviewer mentioned several lightweight segmentation networks ([6], [7], [8], [9], [10]) that were not fully explored in our initial submission. Our clarification is as follows:

• [6] (SlimUNETR, Pang et al., 2023): To strengthen our evaluation, in Table R2 above (also in Tables 3, 12 of the revised draft), we have now added results for SlimUNETR (a recent lightweight 3D segmentation method). Table R2 shows that UltraLightUNet3D-S achieves 10.19% higher DICE score on Task05 Prostate, 0.17% higher on FETA, 1.47% higher on Synapse 8-organ, and 2.25% higher on Synapse 13-organ while using 9.9x fewer #parameters and 5.9x fewer #FLOPs than SlimUNETR. Larger variants like UltraLightUNet3D improve further, with UltraLightUNet3D-M showing a total improvement of 12.50% on Task05 Prostate, 1.42% on FETA, 2.16% on Synapse 8-organ, and 4.90% on Synapse 13-organ compared to SlimUNETR. These results demonstrate UltraLightUNet3D’s significant improvements in performance while maintaining exceptional computational efficiency and cost.

• [7] (CMUNeXt): Already included in our initial submission (see Table 1, 2, 11 in our initial submission); again, UltraLightUNet demonstrates its superior performance and efficiency compared to CMUNeXt.

• [8], [9], [10]: These methods are for 2D segmentation and do not extend to volumetric 3D segmentation tasks. While they provide valuable contributions for 2D segmentation tasks, their lack of 3D applicability makes them less relevant for a direct comparison with UltraLightUNet 3D version.

With the addition of SlimUNETR in our rebuttal and revised manuscript, along with the necessary comparisons for 2D methods, we believe our paper provides comprehensive coverage of both 2D and 3D lightweight segmentation networks.

We thank the reviewer for their detailed feedback and for pointing out areas for improvement. Below, we address all the comments and clarify our approach:

Q1. The overall novelty is low. The author mainly proposed two modules, CMFA and MKDC modules. MKDC just applied several depth-wise convolutional layers to extract features from different channels. This idea has already been proposed in the Scale-Aware Modulation Meet Transformer [1]. MKDC module employed average and max pooling, and convolution layers, which are the channel-wise and spatial-wise attention. Many various attention-based modules have been proposed between 2018 and 2020 (like the CBAM [2]). The overall network is similar with [3] [4].

Response:

UltraLightUNet introduces significant contributions both at architecture and module levels. At the architecture level, UltraLightUNet offers a fully integrated, lightweight encoder-decoder design, specifically tailored for resource-constrained scenarios, thus achieving competitive segmentation performance across both 2D and 3D tasks. At the module level, UltraLightUNet incorporates innovative designs like the Multi-Kernel Inverted Residual (MKIR) and Multi-Kernel Inverted Residual Attention (MKIRA) blocks, enabling efficient feature extraction and refinement with minimal computational cost. Below, we provide details on these contributions.

1. New End-to-End Lightweight Design

UltraLightUNet is a novel encoder-decoder architecture built entirely from scratch to ensure extreme lightweight efficiency. It is specifically designed for resource-constrained scenarios, including real-time medical diagnostics and point-of-care applications. Both the encoder and decoder are specifically optimized with novel lightweight modules, eliminating the need for pre-trained components while maintaining competitive performance across diverse tasks.

• Encoder Design: The encoder is built using our proposed Multi-Kernel Inverted Residual (MKIR) Block, which performs efficient feature extraction through a combination of multi-kernel depth-wise convolutions and an inverted residual structure. This ensures adaptable feature extraction with minimal computational overhead, supporting both local (small kernels) and global (large kernels) context extraction.

• Decoder Design: The decoder is constructed with our novel Multi-Kernel Inverted Residual Attention (MKIRA) Block, which combines Convolutional Multi-Focal Attention (CMFA) for local attention and MKIR for multi-kernel refinement. Additionally, the decoder employs a Grouped Attention Gate (GAG) for efficient skip connection aggregation and the simple bilinear upsampling, thus ensuring lightweight refinement and reconstruction of segmentation outputs.

With this integrated design, UltraLightUNet achieves unmatched efficiency with only 0.316M parameters and 0.314 GFLOPs for its 2D base model, and 0.453M parameters and 3.42 GFLOPs for its 3D base model. This is a significant advancement compared to related methods like SAMT the reviewer mentions (32M parameters, 7.7 GFLOPs), CASCADE (34.12M parameters, 7.62 GFLOPs), and EMCAD (26.76M parameters, 5.6 GFLOPs), all of which depend on pre-trained encoders or computationally expensive modules. In contrast, UltraLightUNet’s lightweight design specifically addresses the computational constraints of real-world applications in point-of-care scenarios without sacrificing performance.

Thanks for authors' response. I will not change my score due to the low novelty, low impact to the medical image segmentation, and too lower baseline results.

"The reported results for Swin Unet and TransUNet in our manuscript were taken directly from the CASCADE paper [3], ensuring consistency across all the reported baselines."

It is not fair to copy results of baselines from other papers, and it is better to check the original paper to utilize the hyper-parameters in their original papers. It is not a good way to train your model to the optimal one while not use optimal hyper-parameters to train baseline models. I am very confused why you copied results from the CASCADE paper. This CASCADE paper is not a benchmark paper, so copying results from this paper will not ensure the consistency. These baselines were not proposed in the CASCADE paper, and copying results from this paper will lead to lower segmentation performance and unfair comparison. These baselines have not reached their highest performance, and they still have large potential to reach the higher performance. Therefore, it cannot demonstrate the superiority of the UltraLightUNet.

Addressing "Application from N to N+1"

We respectfully disagree with the characterization of our work as an incremental application. While our design leverages depth-wise convolutions, it introduces novel mechanisms (based on a new mathematical basis in Equation 2) such as multi-kernel refinement for adaptable feature extraction and attention, as well as seamless 3D extensions of all modules. These innovations improve the state-of-the-art efficiency, which is critical for practical deployment in clinical settings which are scarce in resources.

Moreover, our results demonstrate that UltraLightUNet outperforms (by orders of magnitude!) several heavier and lighter models in terms of computational cost and segmentation accuracy across diverse datasets, including both 2D and 3D tasks. This underscores the impact of our approach in advancing lightweight segmentation methods, particularly for real-time medical diagnostics.

Broader Context and Future Impact

Our work is part of a growing trend in computer vision in general, and biomedical imaging in particular, toward developing ultralightweight models that can deliver high precision in real-time tasks. Examples of this trend include:

• MobileNets (Howard et al., 2017) and EfficientNet (Tan et al., 2019) in computer vision,

• TinyBERT (Jiao et al., 2019) in language processing,

• UNeXt (Valanarasu et al., 2022), EGE-UNet (Ruan et al., 2023), and Rolling-UNet (Liu et al., 2024) in biomedical imaging.

All these lightweight architectures made headlines in the field by making possible vision/language/imaging tasks with significantly less resources. In other words, “less is more” is the new name of the game as this focus on efficiency is a paradigm shift in the making. We believe that the significance of this type of work will only increase in future years as the demand for efficient, real-time models will continue to grow. From this perspective, UltraLightUNet’s contribution is particularly timely, by addressing the need for accurate, lightweight segmentation solutions in resource-constrained environments.

We hope these clarifications address the reviewer’s concerns and highlight the importance and relevance of our contributions to the field. Thank you again for your thoughtful feedback and consideration.

C1. The novelty is low since the way to solve the problem has been widely explored and is the same, such as employing depth-wise convolution for lightweight design and splitting channels for isolated convolutions. Thus, putting much efforts on applying the exactly same way to solve the same problem is not very interesting, and this application from N to N+1 does not make impact on the medical image segmentation tasks. The architectural design cannot be considered as a novel design since the overall design (U-shaped encoder-decoder) is always used in the medical image segmentation. Incorporating several modules into this architecture does not revolutionize the architectural design.

Response:

We thank the reviewer for their follow-up comments and the opportunity to clarify our contributions further. Below, we provide a detailed response to these new concerns.

Addressing Novelty in Architectural Design

Yes, the U-shaped encoder-decoder design is a well-established framework in medical image segmentation, originating with the seminal U-Net paper in 2015 (Ronneberger et al.). However, we believe that using the U-Net popularity as a penalizing argument for further creative improvements of this basic architecture is too reductionist in nature and actually disconnected from the reality in this area. In fact, as evidenced by a wide body of literature, this foundational U-design has seen continuous improvements over the years to address specific challenges in accuracy and computational efficiency. Notable examples include Attention UNet (Oktay et al., MIDL 2018), UNeXt (Valanarasu et al., MICCAI 2022), TransUNet (Chen et al., Medical Image Analysis 2024), EMCAD (Rahman et al., CVPR 2024), 3D UX-Net (Lee et al., ICLR 2023), and Swinunetr-v2 (He et al., MICCAI 2023), among others. These works improve over the basic U-Net architecture and highlight the significance of improving U-shaped architectures rather than aiming only for entirely revolutionary designs, which are rare. In fact, one of the most enduring contributions of the UNet architecture is precisely this wide-open area of research it enables which provides ample opportunities for further improvements to the SOTA.

Our contribution aligns perfectly with this ongoing trend, but with a distinct focus on addressing extremely low computational costs while maintaining high segmentation accuracy. Unlike prior works that often increase architectural complexity to improve accuracy (e.g., TransUNet, 3D UX-Net, SwinUNETR-v2), UltraLightUNet provides a new type of solution focused on extreme efficiency, thus making it uniquely suited for real-time, resource-constrained scenarios such as point-of-care diagnostics. From this perspective, the UltraLightUNet architecture redefines the SOTA in image segmentation so this is why we believe our contribution is worthwhile and we should not be penalized for providing better results than SOTA.

Advancing Lightweight Design in Biomedical Imaging

Yes, depth-wise convolution and channel splitting are commonly used. However, it is essential to recognize that leveraging established concepts in novel ways to meet specific challenges and redefine the SOTA is a widely accepted and desirable approach in the field. For instance:

• MobileNet (Howard et al., 2017) and EfficientNet (Tan et al., 2019) have successfully advanced computer vision using depth-wise convolutions (an idea already known at that time) for lightweight designs.

• In biomedical imaging, UNeXt (Valanarasu et al., 2022), EGE-UNet (Ruan et al., 2023), and Rolling-UNet (Liu et al., 2024) have similarly employed lightweight modules for efficient segmentation, so simply improving SOTA with better ideas.

UltraLightUNet contributes to this trend in research by introducing the new Equation 2 (reproduced in our first response of this rebuttal) that makes the mathematics of Multi-Kernel Inverted Residual (MKIR) and Multi-Kernel Inverted Residual Attention (MKIRA) blocks far more efficient by going beyond standard depth-wise convolutions by enabling adaptable feature extraction across diverse spatial contexts. Additionally, our integration of these modules into both 2D and 3D designs extends the applicability of lightweight models to volumetric medical imaging, a challenging and underexplored area. So, it is the new mathematical basis and the extended scope of our research that make our contribution worthwhile.

Thanks for your response. It seems that you misunderstood what I wanted to deliver. I read your responses carefully, but your responses didn't address my major concern.

(1) The main contributions of this manuscript was proposing several modules, but these modules are similar to modules from the paper EMCAD, as another reviewer mentioned. Although you provided details and tried to clarify they were not 100% same, I believe they are at least 90% similar. (2) You said your architecture was novel since you incorporated your modules into the encoder. Unfortunately, incorporating modules into both the encoder and the decoder is not novel and is a marginal contribution since almost everyone is trying to incorporating their modules into both the encoder and the decoder. In other words, incorporating their modules into U-Net is not novel, and this is a common design. Many works have done it. (3) Using depthwise convolutions to reduce the Params and FLOPS has been explored from ConvNext and UX Net several years ago. These two are just examples, and these are many other networks are trying to use the depth-wise convolutions to improve efficiency after these two works. Thus, this idea is not new, and this idea in this manuscript doesn't provide new theoretical insight to others since this idea has been explored from several years.

Most importantly, what I wanted to delivered was from the higher level, not about the details of your modules. You mentioned your designs in architectures and modules, and they may be novel to some other conferences. However, in ICLR, we would like to see some works that propose fundamentally different solutions and theoretically sound methods. Specifically, in the field of medical image segmentation, U-Net and Vision Transformers have been widely explored, so we would like to see some other newly-proposed architecture instead of U-Net and Vision Transformers, or some architectures that have not been applied to medical image segmentation. You utilized the depth-wise convolution to reduce Params and FLOPs, but it has been widely explored after ConNext and UXNet. The biggest selling point of your manuscript is the reduction of Params and FLOPs which benefits from the utilization of depth-wise convolutions, but this solution is not fundamentally different from others. They incorporated depth-wise convolutions into the decoder, and you incorporated it into both the encoder and decoder. These two designs are fundamentally same. We would like to see if you can propose new convolutional operations instead of depth-wise convolutions to reduce Params and FLOPs. In ICLR, I believe improving performance is not the first priority, since improving performance is not very hard, and performance will be influenced by the training strategies and computational resources. Instead, we would like to see solutions that tackle problems in a fundamentally different way, inspiring other readers' to explore this field deeply. Your work may achieve a promising result from the engineering side. However, it doesn't tackle this problem fundamentally, or provides more theoretical insights to others.

Thanks for authors' responses.

(1) Like another reviewer jeKK said, the modules in this manuscript is same with modules in the paper EMCAD, so there is no module novelty. Like your title said, the model in this manuscript is still using U-Net architecture, so there is no architectural novelty as you mentioned. (2) The improvement of this manuscript is low, and contributions of this manuscript is low. The major contributions of this manuscript are mainly built over the paper EMCAD, resulting in a low impact to the field of medical image segmentation. (3) The work in this manuscript is trying to incorporate several previously proposed and widely used modules into a UNet architecture, so it doesn't provide theoretical insights to others.

Overall, I will not change my score, "1 strong reject".

Q4: The proposed method is not compared to other SOTA networks (such as (Azad et al., 2024)) which perform much better over some of the datasets used in the paper (such as ISIC)

The reviewer mentions that UltraLightUNet lacks comparisons to certain SOTA models, such as Deformable Large Kernel Attention.

Response:

• Existing Comparisons: We do already compare UltraLightUNet with multiple SOTA models, including DeepLabv3+, TransUNet, SwinUNet, and lightweight architectures like UNeXt, CMUNeXt, EGE-UNet, Ultra_Light_VM_UNet. Without exception, these comparisons demonstrate that UltraLightUNet achieves superior or competitive segmentation accuracy with significantly fewer parameters and lower computational costs.

• Newly Added Comparisons: Per reviewer suggestion, in the revised manuscript, we will include additional comparisons with recent SOTA methods, such as Deformable Large Kernel Attention (Azad et al., 2024). Below, we provide a detailed comparison of skin lesion segmentation on the ISIC 2018 and BUSI datasets.

Table R2: Comparison of UltraLightUNet with Deformable Large Kernel Attention (DeformableLKA). We present the DICE scores (%) on our data-splits with an input resolution of 256 $\times$ 256, while optimizing the hyper-parameters of DeformableLKA.

Again, Table R2 shows that UltraLightUNet provides very competitive results using a fraction of #Params and #FLOPS compared to the DeformableLKA approach. Specifically, DeformableLKA, with a pretrained encoder, achieves the highest DICE score on ISIC 2018 (90.34%) and BUSI (79.01%) but requires 3,805x more #Params and 1,001x more #FLOPs than UltraLightUNet-T (0.027M parameters and 0.026G FLOPs). Without pretraining, DeformableLKA’s DICE scores drop by 2.17% on ISIC 2018 and 4.39% on BUSI, thus falling below the DICE scores of our UltraLightUNet-T (88.19% and 75.64%).

In contrast, UltraLightUNet-M which does not rely on pretraining, delivers very competitive DICE scores: 89.09% (ISIC 2018) and 78.27% (BUSI), thus narrowing the gap to just 1.25% on ISIC 2018 and 0.74% on BUSI compared to pretrained DeformableLKA.

We are genuinely surprised by the ethical concerns raised regarding our UltraLightUNet paper. To clarify unequivocally and categorically, our ICLR submission is neither an act of plagiarism nor a duplicate submission of the EMCAD (Rahman et al., 2024). While both papers focus on improving efficiency in medical image segmentation, they have fundamentally different goals, methodologies, and contributions.

Below, we address each and every concern and clarify the originality and contributions of our UltraLightUNet submission.

Q1. Despite the paper presents diverse experiments and ablation studies, I see a very close similarity to a CVPR 2024 paper, named EMCAD (Rahman et al., 2024), which I detail in the next parts

Response: We respectfully disagree with this assessment. While UltraLightUNet and EMCAD share a U-shaped architecture, which is widely used for medical image segmentation, they differ significantly in their architectural philosophy, key innovations, and target use cases (as summarized in Table R1 below):

Table R1: Differences between UltraLightUNet and EMCAD

We can also illustrate these differences using an analogy from transportation and urban planning:

UltraLightUNet is akin to a compact electric scooter built for quick, efficient navigation in crowded city streets. It’s lightweight, maneuverable, and optimized for short, real-time trips (analogy to real-time diagnostics) in constrained environments where heavy vehicles (complex models) can’t operate effectively. The focus is on minimalism and efficiency, designed to get the job done swiftly without excess fuel consumption (parameters and FLOPs in our case of resource-constrained scenarios).

In contrast, EMCAD is like a comprehensive highway system designed to handle various types of traffic (cars, buses, trucks) efficiently, with multi-lane roads (multi-scale attention modules) and advanced traffic management (hierarchical feature refinement) to balance local (urban streets) and global (interstate highways) needs. It aims to provide a versatile, all-purpose solution that works well in different terrains and conditions.

Response:

We thank the reviewer for their follow-up comments and the opportunity to clarify our contributions further. Below, we address this reviewer’s concerns directly and highlight the unique aspects of our work.

1. Novelty of UltraLightUNet Modules

Multi-Kernel Inverted Residual (MKIR) Block

The MKIR block introduces a novel feature extraction mechanism based on our multi-kernel trick (i.e., MKDC) in Equation 2 (reproduced below):

$MKDC(x) = CS\left(\sum_{k \in K} DWCB_k(x)\right)$

Where $DWCB_k(x) = ReLU6(BN(DWC_k(x)))$. Here, $DWC_k(x)$ is a depth-wise convolution with kernel $k$. Of note, our MKDC supports both $k_1=k_2$ (same-size kernels, e.g., $[3 \times 3,3 \times 3,3 \times 3]$, $[5 \times 5,5 \times 5,5 \times 5]$) and $k_1 \neq k_2$ (different-size kernels, e.g., $[1 \times 1,3 \times 3,5 \times 5]$), for $k_1, k_2 \in K$. This flexibility in supporting both identical and different scale kernels distinguishes the MKDC from the EMCAD’s MSDC block (see Equation 5 in the EMCAD paper):

$MSDC(x) = \sum_{k \in K} DWCB_k(x)$

which is restricted only to multi-scale designs ($k_1 \neq k_2$, e.g., $[1 \times 1,3 \times 3,5 \times 5]$). This shows the new contribution MKIR brings compared to MSDC, mathematically speaking; this difference is the very basis for the excellent results of the UltraLightUNet architecture.

Indeed, by enabling adaptable kernel configurations, MKIR offers efficient and versatile feature extraction tailored to application-specific needs, thus reducing computational cost drastically while achieving high segmentation accuracy. An illustrative example, various statistics, and empirical evidence supporting our claims are described next.

Explaining Multi-Kernel vs. Multi-Scale with an Illustrative Example:

Let us start with an analogy: Imagine we are drawing on a piece of paper with different types of paintbrushes: small brushes, big brushes, and a set of both small and big brushes. Now, let's say we want to color objects of different sizes, like:

• Small objects, such as tiny dots.
• Large objects, like big circles.
• Mixed objects, where we have both tiny dots and big circles.

The Multi-Kernel Scenario (akin to this ICLR paper): In the multi-kernel scenario, we can choose our brushes based on what we need:

• Only small brushes for tiny dots.
• Only big brushes for big circles.
• A mix of small and big brushes for both tiny dots and big circles.

This flexibility lets us adapt our tools (kernels) for the specific task at hand. In other words, we might say, "we’ll use just small brushes for this drawing because it’s all tiny dots," or "Let’s use a mix because we need to cover both."

The Multi-Scale Scenario (akin to the EMCAD paper): In the multi-scale scenario, we must always use a mix of small and big brushes together, no matter what. Whether we have tiny dots, big circles, or both, we must use the mix of paintbrushes. This may work well for some cases, but it’s clearly less flexible because we can’t decide to use just one type of brush when we have only tiny or big objects.

How this Analogy Applies to Image Segmentation: In medical imaging:

• Small objects: Tiny tumors or small lesions.
• Large objects: Big organs like the liver or spleen.
• Mixed objects: Both small lesions and large organs in the same image.

6. Comprehensive Ablation Studies

• We emphasized the contributions of individual modules (e.g., MKIR, MKIRA, CMFA, GAG) using detailed ablation studies (Tables 4 and 5 in the initial submission). Results show that:
  • MKIR and MKIRA are the most critical for accuracy improvements, especially on datasets like BUSI (from 72.41% to 76.61%).
  • Combining all modules yields the best overall performance (78.04% on BUSI).
• These studies validate the design of UltraLightUNet and will be further clarified in the revised manuscript.

7. Clarity on Model Variants

• We clarified the differences among UltraLightUNet-T, S, M, and L:
  • Variations in channel dimensions across encoder-decoder stages (e.g., T: (4, 8, 16, 24, 32); M: (32, 64, 128, 192, 320)).
  • Scalability analysis (Tables 9 and 10) validates the performance improvements with increasing channel counts.

8. Limitations and Future Directions

• We added a dedicated subsection in Appendix A.14 to discuss the limitations of UltraLightUNet:
  • Slightly lower performance on highly complex datasets compared to heavyweight SOTA models.
  • Trade-offs between computational efficiency and segmentation accuracy.
• We also outlined future directions, including hybrid architectures, self-supervised pretraining, and extensions to tasks like image reconstruction, enhancement, and denoising.

Key Revisions

Based on the rebuttal, we incorporate the following changes:

1. Expanded Introduction and Related Work:

   • Discuss recent lightweight (SlimUNETR) and relevant (DeformableLKA) methods.
   • Clearly differentiate UltraLightUNet from EMCAD, ConvNeXt, SAMT, CASCADE, and SFCNs.
   • Highlight the novelty of UltraLightUNet’s unified multi-kernel approach.

2. Revised Methodology:

   • Emphasize the theoretical basis for multi-kernel ($k_1 = k_2$ and $k_1 ≠ k_2$) vs. multi-scale ($k_1 ≠ k_2$) approaches and its adaptability across diverse tasks.
   • Modify Figure 2 to reflect the conceptual distinctions.
   • Clarify the differences among UltraLightUNet-T, S, M, and L.

3. Updated and Added Tables and Figures:

   • Add results for DeformableLKA (Table R2) and SlimUNETR (Table R4) to Tables 1, 3, 12, 13, and 14 in our revised manuscript.
   • Include results on new datasets (Table R5) to Table 13 in our revised manuscript.
   • Add activation heatmaps (Figure R1) to enhance interpretability discussions (Figure 3 in our revised manuscript).
   • Add training/inference metrics (Table R3) to Table 14 in Appendix A.13.
   • Updated the DICE score of SwinUNet in Table 2 on Synapse dataset to reflect the original authors DICE score as per the reviewer suggestion.

4. Added Limitations and Future Directions:

   • Include a discussion on limitations and future directions in Appendix A.14.

We believe these revisions comprehensively address the reviewers' comments, strengthen the manuscript, and clarify UltraLightUNet’s significant contributions to lightweight medical image segmentation. Thank you for your valuable feedback, which has greatly enhanced the rigor and clarity of our work. We look forward to submitting the revised version.

References

Rahman et al., 2024. Emcad: Efficient multi-scale convolutional attention decoding for medical image segmentation. CVPR (pp. 11769-11779).

Sandler wt al., 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. CVPR (pp. 4510-4520).

Hu et al., 2018. Squeeze-and-excitation networks. CVPR (pp. 7132-7141).

He et al., 2016. Deep residual learning for image recognition. CVPR (pp. 770-778).

Tan et al., 2019, May. Efficientnet: Rethinking model scaling for convolutional neural networks. ICML (pp. 6105-6114).

Azad et al., 2024. Beyond self-attention: Deformable large kernel attention for medical image segmentation. WACV (pp. 1287-1297).

Lin et al., 2023. Scale-aware modulation meet transformer. ICCV (pp. 6015-6026).

Woo et al., 2018. Cbam: Convolutional block attention module. ECCV (pp. 3-19).

Rahman et al., 2023. Medical image segmentation via cascaded attention decoding. WACV (pp. 6222-6231).

Cao et al., 2022. Swin-unet: Unet-like pure transformer for medical image segmentation. ECCV (pp. 205-218).

Pang et al., 2023. Slim UNETR: Scale hybrid transformers to efficient 3D medical image segmentation under limited computational resources. IEEE Transactions on Medical Imaging

Tang et al., 2024. Cmunext: An efficient medical image segmentation network based on large kernel and skip fusion. ISBI (pp. 1-5).

Yang et al., 2023. UcUNet: A lightweight and precise medical image segmentation network based on efficient large kernel U-shaped convolutional module design. Knowledge-Based Systems, 278, 110868.