LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation

We sincerely thank reviewer for the constructive feedback. Our responses are given below.

Q1.1. How do learnable weights compare with other dynamic loss weights methods?

Response: To our knowledge, there is no prior method that dynamically balances dozens of multi‑scale, single‑task supervision losses the way LoMiX does. Prior work falls into two categories:

• Deep supervision in segmentation (fixed or hand‑scheduled weights): Classical deeply‑supervised/multi‑scale segmentation papers use side heads but keep their weights constant or manually decayed, e.g., U‑Net++ [7] and DSN [8]. None of these adapt weights data‑dependently across combinatorial multi-scale heads.
• Dynamic loss weighting designed for multiple (heterogeneous) tasks: GradNorm [9] and Uncertainty weighting [10] address multi‑task learning where the number of losses is small (typically 2–5). Their normalization schedules or gradient‑matching heuristics may not scale well to dozens of highly correlated single‑task losses (one per fusion path).

LoMiX fills this gap by reformulating the single‑task, multi‑scale supervision as a multi-operator, combinatorial, fully learnable problem at logit/loss level. Each fusion path gets its own Softplus weight, so the optimizer can automatically suppress the unhelpful paths and emphasize the useful ones. In our ablations, fixed/uniform weights baselines underperform our NAS‑inspired weighting significantly (Fig. 4 and Supplementary Table S7). If a truly comparable single‑task, multi‑scale dynamic weighting method exists, we will gladly cite and benchmark it.

Q1.2. How LoMiX different from other late-fusions?.

Response: This is how LoMiX is different:

• Where/when it operates: We fuse only during training, thus having zero extra parameters or latency overhead at inference. Late‑fusion papers [1,2] keep their fusion blocks at inference, thus adding overhead.
• What is fused: We fuse multi-scale decoder logits, not high‑dimensional features, using multiple operators (Add, Mult, Concat, AWF) over all non‑empty subsets, yielding a combinatorial, diverse set of supervision signals rather than a single fixed fusion path.
• Why we fuse: Our goal is a stronger, more diverse supervision, i.e., an implicit ensemble that improves gradients (especially in data‑scarce regimes), not a deeper architecture.
• How do we optimize: Each original/fused logit gets a NAS‑inspired, Softplus loss weight learned end‑to‑end; prior works [1,2] do not auto‑weight dozens of fusion paths.
• Empirical evidence: Operator/subset ablations (Fig. 3 and Supplementary Table S6) and weight‑dynamics plots (Figs. S1-S8) show that different fusion types/resolutions do matter, and the learned weights self-rebalance over training.

To sum up, LoMiX is not a late fusion block with off‑the‑shelf operations; instead, it is a new combinatorial, and automatically weighted supervision methodology at training-time. This clearly differentiate it from methods that use similar components (names), but with completely different purposes.

Q2 & W1. Comparison methods and restructuring main table.

Response: The confusion stems from considering LoMiX a PVT‑EMCAD add‑on. In fact, LoMiX is a new architecture‑agnostic supervision methodology at training‑time, so we must show that it works across multiple networks, not just one.

Fig. 5 and Table S8 already show that LoMiX scales well across different CNN/Transformer encoders with the EMCAD decoder. To restructure Table 1 as per-network progression, we run new experiments with different supervision across representative CNN, MLP and Transformer hybrid networks. The results show that LoMiX achieves the best DICE for every network (see Table R6). We will add the full restructured Table 1 in the revised paper. This will directly show "what does LoMiX add beyond existing supervisions?"

Table R6: Synapse 8-organ segmentation with Last Layer (LL), Deep Supervision (DS) [7], MUTATION [18], and LoMiX. DICE scores (%) reported for Gallbladder (GB), Left kidney (KL), Right kidney (KR), Pancreas (PC), Spleen (SP), and Stomach (SM). We report results of only three networks here for space limits.

Q3 & W2. Novelty and claim of this paper...

Response: We respectfully disagree with this assessment, as we propose:

• A new supervision methodology, not a new architecture. Prior works [1, 2] aggregate high-dimensional features inside the network and keep those aggregation steps at inference. In contrast, LoMiX operates at the logit/loss level, during training only. We synthesize intermediate resolution prediction maps (class logits) from existing multi-stage/resolution decoder outputs only during training, thus having zero extra parameters and latency overhead at inference. Our focus is to optimize supervision for any U-shaped network.
• Operator-diverse combinatorial logit fusion: Prior works [1,2] fuse a fixed set of multi-scale wide feature tensors. In contrast, LoMiX systematically considers all non‑empty subsets of multi-scale decoder logits and applies a set of operators (Add, Mult, Concat, AWF) to generate a rich set of intermediate resolution predictions. This exhaustive, resolution- and operator-diverse logit space has never been explored for segmentation supervision. This makes our contribution unique and worthwhile.
• Fully automatic, NAS-inspired loss weighting: Rather than hand‑tuning or using uniform loss weights, we introduce a NAS‑inspired Softplus weighting mechanism that learns how much each original/fused logit should contribute to the loss. This way, dozens of fusion paths are jointly optimized, thus making the combinatorial space tractable.
• Data-scarce medical segmentation: Our largest gains appear where supervision quality matters most, i.e., in limited-data settings which is common in practical medical imaging. Prior work [1,2] targets large-scale natural-image detection with abundant labels. We show that rethinking supervision (instead of architecture) yields consistent DICE improvements across diverse medical datasets/networks with limited annotated data.
• Thorough analyses: We provide operator/subset ablations (Fig. 3 and Supplementary Table S6), weight-dynamics curves (Supplementary Figs. S1-S8), per-organ results (Tables 1-2, and Supplementary Tables S1, S3, S5-S8), and cross-architecture studies (Fig. 5 and Supplementary Table S8). This demonstrates generality and interpretability rather than an on-off tweak.

To mitigate reviewer concerns, we will discuss [1,2] in the related work, and rephrase the claim in line 121: "In contrast, to the best of our knowledge, LoMix is the first approach to enable fully learnable multi-scale logit fusion for improved supervision in segmentation."

W3. Compare NAS-based loss weights with other dynamic loss methods.

Response: The "dynamic loss" schemes (GradNorm [9], Uncertainty weighting [10], etc.) target multi‑task settings with a few heterogeneous losses. In contrast, our LoMiX tackles a single task (segmentation) with dozens of highly correlated, multi‑scale logit losses synthesized using our multi-operator combinatorial fusion. Consequently, we designed a new NAS‑inspired Softplus weighting mechanism that (i) assigns each path an independent positive scalar, (ii) requires no global normalization or manual weights, and (iii) scales well to many multi-scale loss signals, which is exactly what we need in our setup. To keep the evaluation focused, we compared against the closest relevant baselines for our setting: multi‑scale/deep supervision with fixed or uniform weights (e.g., deep supervision [7], MUTATION [18]). Those baselines directly isolate the effect of making supervision learnable, which is our core contribution.

In principle, there is no fundamental barrier to plugging in multi‑task dynamic methods; however, they are simply not the right comparisons for a single‑task, multi-scale loss supervision problem. If the reviewer still deems such a comparison essential, we will include it in the revised paper.

References

[1] Liu, S., et al. Path aggregation network for instance segmentation. CVPR, 2018.

[2] Zhao, Q., et al. M2det: A single-shot object detector based on multi-level feature pyramid network. AAAI, 2019.

[7] Zhou, Z., et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. TMI, 2019.

[8] Lee, C.Y., et al. Deeply-supervised nets. AIS, 2015.

[9] Chen, Z., et al., Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. ICML, 2018.

[10] Kendall et al., Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. CVPR, 2018.

Thanks for the detailed explanation. The author provides a clear explanation of the late fusion part (which is not a new module).

However, I still find it hard to identify the fundamental novelty of the learnable weights of the loss term. Basically, what the author introduces in this paper is a method to get $TotalLoss = w_1 * loss_1 + w_2 * loss_2 + w_3 * loss_3 + ...$, where the $w_{i}$ are learnable parameters (see eq. 6 and eq. 9), and the losses come from off-the-shelf operators. The whole process is trivial.

On the other hand, even though the author claims to be handling a 'single task (segmentation) with dozens of highly correlated, multi-scale logit losses synthesized using our multi-operator combinatorial fusion,' the weighting process does not provide a specific design for that. The author just treats the problem as a multi-loss problem, regardless of whether the losses are 'highly correlated' or 'multi-scale.'

(i) It 'assigns each path an independent positive scalar.' Since the paper is dealing with multiple loss terms, I think an independent positive scalar is the default option. (ii) requires no global normalization or manual weights. " This is basically what nn.Parameters are doing. (iii) It 'scales well to many multi-scale loss signals.' I can't see a specific multi-scale loss weight design in Eq. 6, 7, 8, or 9."

Overall, based on the parts of "A new supervision methodology", "Operator-diverse combinatorial logit fusion", and "Thorough analyses", I would raise my rating to 3. Still, I think the whole framework is trivial for NeurIPS.

We sincerely thank the reviewer for the constructive feedback. Below, we address each concern and outline how we plan to incorporate these suggestions in our revised manuscript.

Q1: The paper does not discuss the convergence of multi-scale fusion optimization. For example, are the weights of all fusion operations eventually stable? Are there redundancies?

Response: Our multi‑scale fusion does converge and self‑suppresses redundancies. As the Softplus weights are co‑optimized with the segmentation model, absolute values drop with the overall loss, so after a short warm‑up their relative ordering stabilizes (see Supplementary Figs. S1–S8). Two consistent patterns emerge:

• By scale: Low‑resolution, high‑level logits (early decoder stages) are driven to relatively much smaller weights, while high‑resolution, fine‑grain logits (later stages) retain larger weights, indicating that less informative logits are automatically down‑weighted.

• By operator: AWF and Add fusions receive comparatively higher weights, Mult gets moderate weights, and Concat gets the lowest weights in later epochs (see Supplementary Fig. S1–S8). Still, even these low‑weighted Concat fused logits matter; our ablations (Fig. 3 and Supplementary Table S6) show that they supply complementary segmentation cues that improve DICE despite their small scalar values.

Q2. Different image modalities (e.g., ultrasound vs. CT) may require different fusion strategies, but the paper does not discuss parameter migration or modality adaptation approaches.

Response: We agree that ultrasound, CT, MRI, and endoscopy have very different characteristics. In fact, LoMiX is designed precisely to learn such differences rather than hard‑code a single fusion recipe. Our Softplus loss weights are co‑optimized with the segmentation network, so the model automatically explores the full fusion space and down‑weights the less useful fused logits while relatively upweighting the informative ones for each dataset/modality. In practice, all weights numerically shrink as the overall loss decreases, but their relative pattern stabilizes: low‑resolution/early‑stage logits and certain operators (e.g., Concat) receive relatively much smaller final weights, while high‑resolution logits and AWF/Add remain influential; this is evidence of data‑driven adaptation to various datasets/modalities (see Supplementary Figs. S1-S8).

Because this weighting is learned jointly, no separate “parameter migration” or modality-specific tuning is needed: training (or fine‑tuning) on a new modality simply yields a different weight configuration. We already demonstrate this across heterogeneous modalities, such as CT organs (see Table 1), cardiac MRI (Table 2), ultrasound (Table S2), skin lesions (Table S2), and colon polyps (Table S4), without changing the architecture or hand‑tuning weights. We will make this modality‑adaptation mechanism more explicit in the camera-ready version. Equally important, LoMiX works at training‑time, so clinical deployment is unaffected (i.e., zero extra parameters or latency overhead during inference).

W1. Medical image analysis often requires 3D segmentation, but the paper has not yet verified the scalability and capability of LoMix on 3D architectures.

Response: Generalizing LoMiX to 3D is straightforward: simply replace the 2D convs and bilinear upsampling with 3D convs and trilinear upsampling, everything else remains unchanged. This is because we fuse predictions (class logits), not wide feature tensors: our goal is to synthesize intermediate "pseudo‑ensembled" predictions during training for richer supervision, with zero inference overhead.

Adding all non‑empty subsets of the $L$ decoder predictions introduces only $2^L−L−1$ extra logit maps (e.g., $11$ for $L=4$, $26$ for $L=5$); since U‑shaped networks rarely exceed five stages, these maps are finite, lightweight, and used only during training, so the incremental training compute/memory cost remains feasible and inference is still zero‑overhead. When GPU memory is low, either in 2D or 3D, standard optimizations such as gradient checkpointing or caching logits on CPU further reduce compute requirements without altering the algorithm, thus keeping LoMiX practical on modest hardware (< 5 GB extra GPU memory required for backpropagation to process a 96×96×96 volume with a four-stage network and 9 output classes). To show the feasibility of LoMiX with 3D segmentation networks, the priliminary results of 3D MedNeXt with LoMiX are reported in Table R4 below.

Table R4: MedNeXt's performance on 3D Synapse 8-organ segmentation with Last Layer (LL), Deep Supervision (DS), MUTATION, and LoMiX. DICE scores (%) are reported for Gallbladder (GB), Left kidney (KL), Right kidney (KR), Pancreas (PC), Spleen (SP), and Stomach (SM). Our results (in bold) demonstrate that LoMiX achieves the best average DICE and HD95 scores among all supervisions.

W2: The experiments were based only on publicly available benchmarks, and the generalization ability of LoMix on the data from different acquisition devices or hospitals has not been tested, which may conceal the risk of overfitting.

Response: We agree that clinical deployment demands robustness across scanners and sites. Our current evaluation already addresses overfitting risk by spanning multiple, heterogeneous public datasets and modalities (i.e., abdominal CT, cardiac MRI, breast ultrasound, dermoscopy, colonoscopy), each collected with different protocols and devices. Yet LoMiX improves on every dataset without manual tuning.

Because LoMiX is training‑only, parameter‑efficient, and does not increase model capacity (e.g., parameters and FLOPs), it actually behaves like a regularizer. In fact, LoMiX implicitly ensembles diverse multi‑scale logits only during training, thus reducing the chance of overfitting to dataset‑specific biases.

To make this explicit, in the revised paper, we will include a cross‑dataset experiment (e.g., train on one dataset/hospital, and test on another) as shown in Table R5 below.

Table R5: Cross‑dataset/hospital generalization of LoMiX. Using the PVT‑EMCAD‑B2 network, all models are trained for 200 epochs on the Kvasir polyp‑segmentation training set (900 images); the epoch with the best DICE (%) on the Kvasir validation split (100 images) is saved. Then we evaluated the generalizability on three external test sets (CVC‑ClinicDB, CVC‑ColonDB, ETIS‑LaribPolypDB), without further tuning. LoMiX produces the highest DICE on every dataset which confirm its superior ability to generalize across hospitals and acquisition devices.

Official Response to Reviewer 5cjt

We sincerely thank the reviewer for the constructive feedback. Below, we address each concern and outline how we plan to incorporate these suggestions in our revised manuscript.

W1.1 & Q1. While the NAS-inspired loss weighting partially addresses the supervision imbalance across the combinatorially generated fused logits, all fusion paths are still computed and backpropagated regardless of their utility. This results in significant computational overhead, especially as the number of decoder stages increases. Can the authors justify this design in light of the significant computational overhead?

Response: Thank you for the thoughtful critique. Why keeping all the fusion paths?

We deliberately retain and back-propagate all fusion paths for three intertwined reasons:

• Diversity + automatic weighting: We deliberately enumerate all non‑empty subsets to maximize logit diversity as different operator/subset pairs emphasize different spatial scales and error modes. Our NAS‑inspired Softplus weights then assign distinct, learned contributions to each fusion path; the unhelpful fusion paths are automatically down‑weighted, while the useful ones retain influence. Pruning too early would remove good candidates before the optimizer can assess them.

• Computational overhead is modest and training‑only: All fusions operate on C‑channel class logits (C = #classes, typically small), not on wide feature tensors. Even with 4–5 decoder stages, the extra tensors are at most a few dozen small maps and a few 1×1 (or 1×1×1) convs. In practice, this adds only to the training FLOPs/memory, while inference remains unchanged (i.e., zero overhead). When GPU memory is low for training, standard optimizations such as gradient checkpointing or caching logits on CPU further reduce compute/memory requirements without altering the algorithm, thus keeping LoMiX practical on modest hardware.

• Optional pruning is easy: If desired, one can prune paths whose weights stay near zero after a short warm‑up or sample a top‑K set; this is orthogonal to LoMiX and straightforward to add.

To sum up, computing and backpropagating all paths is a conscious design choice to ensure maximal supervision diversity while letting the NAS‑inspired weighting learn which fused logits matter most. The added cost is small, confined to training, and the result is consistent performance gains without any inference overhead.

W1.2. Moreover, some fusion operators, particularly multiplicative fusion, remain sensitive to low-confidence predictions early in training, potentially introducing unstable gradients before the loss weights converge to suppress them.

Response: We multiply logits, not probabilities, and initialize all Softplus weights to small values, so no path dominates early. If an operator does inject noisy gradients, its weight shrinks. Empirically, our training is stable (see weight dynamics in Supplementary Figs. S1-S8), and removing multiplicative paths can hurt accuracy (see operator ablations in Fig. 3 and Supplementary Table S6), thus indicating they provide complementary information if properly weighted.

W2.1 & Q2. The authors state (line 256) that LoMix helps in segmenting small and challenging abdominal organs, yet no analysis is provided to visualize which fusion types or decoder scales contribute most to those improvements. This would strengthen the claim and help explain LoMix’s behavior. Can the authors provide any analysis or visualizations showing which decoder scales or fusion types (e.g., concatenation, multiplication) are most responsible for these improvements?

Response: We do report already organ‑wise DICE scores for every operator combination in Table S6 and the learned weight dynamics in Supplementary Figs. S1-S8. Together, these results show that paths involving the higher-resolution decoder stages and AWF/Mult operators consistently correlate with the largest gains on small/low‑volume abdominal organs, the scenario highlighted in line 256-257 (Section 4.2).

To make this linkage more clear, we will revise the camera-ready version as follows:

• Add clear cross‑references in the main text from the small‑organ claim to Table S6 and Supplementary Figs. S1-S8.

• Provide qualitative overlays (baseline vs. LoMiX) on representative small organs to visually substantiate the quantitative trend.

• Add visualization of synthesized/fused predictions.

These revisions will directly connect specific fusion paths and scales to the observed boosts on challenging organs, thus strengthening both the interpretability and credibility of our claim.

W2.2. & Q3. Additionally, in Table 2, the performance gain between PVT-EMCAD-B2 with and without LoMix appears modest, and no statistical significance analysis is presented to support the observed differences. Have the authors conducted any statistical tests to confirm that these improvements are significant?

Response: Although the average DICE score gain on PVT‑EMCAD‑B2 looks small, it is consistently and statistically reliable (see Table R2 below). Using a two‑sided Wilcoxon signed‑rank test over 12 test scans (Holm‑corrected), LoMiX is significantly better than every training baseline: MUTATION (p = 0.0122), Deep Supervision (p = 0.0049), and Last Layer (p = 0.0015). We will report these p‑values in main Table 1 to show significance. Again, this improvement comes at zero inference overhead.

Table R2: DICE (%) on 12 test scans of BTCV 8-organ segmentation with PVT‑EMCAD‑B2. p-values (two‑sided Wilcoxon signed‑rank [3], Holm‑corrected [4]) compare each baseline to LoMiX; all are < 0.05, indicating LoMiX’s improvement is statistically significant despite the modest mean gap.

W2.3. & Q4. Also, unlike Table 1 (Synapse), Table 2 omits HD95 and mIoU metrics, which would be relevant for assessing organ boundary quality and overall segmentation agreement, especially since table space permits their inclusion. Why are HD95 and mIoU not reported for the multi-organ segmentation results in Table 2, despite being included for Synapse in Table 1? Including them would allow more complete comparisons.

Response: We agree that HD95 and mIoU are informative for boundary quality and overall agreement. They were omitted from the initial Table 2 to stay consistent with the EMCAD paper’s [5] reporting protocol, but we will definitely include both metrics in the revised paper.

W3. Although the authors acknowledge that LoMix is designed for 2D images and they recognize the weakness, all datasets used in experiments are intrinsically 3D. This mismatch between method and data dimensionality may limit its applicability in clinical practice unless a 3D-compatible version is developed.

Response: LoMiX is dimension‑agnostic: it operates on C‑channel class logit maps at the loss level. Extending LoMix to 3D is straightforward: simply replace Conv2d with Conv3d and bilinear with trilinear upsampling; the fusion/weighting logic remains identical and the cost still scales with the (small) number of decoder stages (rarely exceeding five stages). When GPU memory is low, either in 2D or 3D, standard optimizations such as gradient checkpointing or caching logits on CPU further reduce compute/memory requirements without altering the algorithm, thus keeping LoMiX practical on modest hardware (< 5 GB extra GPU memory required for backpropagation to process a 96×96×96 volume with a four-stage network and 9 output classes).

Of note, we also reported purely 2D tasks (ISIC2018 skin lesions; Kvasir/CVC-ColonDB/ETIS-LaribPolypDB polyps in Supplementary Tables S2 and S4). For the intrinsically 3D datasets, we followed the commonly used memory‑efficient slice‑based protocol as in [5,6], while computing metrics on full volumes. To show the feasibility of LoMiX with 3D segmentation networks, the new preliminary results of 3D MedNeXt with LoMiX are reported in Table R3 below.

Table R3: MedNeXt's performance on 3D Synapse 8-organ segmentation with Last Layer (LL), Deep Supervision (DS), MUTATION, and LoMiX. DICE scores (%) are reported for Gallbladder (GB), Left kidney (KL), Right kidney (KR), Pancreas (PC), Spleen (SP), and Stomach (SM). Our results (in bold) demonstrate that LoMiX achieves the best average DICE and HD95 scores among all supervisions.

References

[3] Wilcoxon, F. Individual comparisons by ranking methods. Biometrics bulletin, 1945.

[4] Holm, S. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, 1979.

[5] Rahman, M.M., et al. Emcad: Efficient multi-scale convolutional attention decoding for medical image segmentation. CVPR, 2024.

[6] Chen, J., et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.

We sincerely thank the reviewer for the constructive feedback. Our responses are given below.

Q1: On line 145: it looks like the number of channels in each decoder layer is constant. Is this a requirement? In that case how would a UNet architecture that increase the number of channels when getting closer to the bottle neck be compatible with the introduced extension?

Response: In our problem definition (Lines 143–145), C is the number of segmentation classes (C = #classes), not the UNet’s internal feature-channel width. LoMix does not require constant channels in the decoder, i.e., a UNet that widens toward the bottleneck is fully compatible. The only requirement is that each decoder stage output is projected onto a C-channel class logit map at a common resolution so these maps can be mixed. We will rephrase the description to make this clear.

Q2: Would it be computationally feasible to generalize the method to a 3D setting? Is there some kind of exponential increase in the number of required fusion maps that gets out of hand?

Response: Yes, generalizing LoMiX to 3D is straightforward: simply replace the 2D convs and bilinear upsampling with 3D convs and trilinear upsampling, everything else remains unchanged. This is because we fuse predictions (class logits), not wide feature tensors: our goal is to synthesize intermediate “pseudo‑ensembled” predictions during training for richer supervision, with zero inference overhead.

Adding all non‑empty subsets of the $L$ decoder predictions introduces only $2^L−L−1$ extra logit maps (e.g., $11$ for $L=4$, $26$ for $L=5$); since U‑shaped networks rarely exceed five stages, these maps are finite, lightweight, and used only during training, so the incremental training compute/memory cost remains feasible and inference is still zero‑overhead. When GPU memory is low, either in 2D or 3D, standard optimizations such as gradient checkpointing or caching logits on CPU further reduce compute requirements without altering the algorithm, thus keeping LoMiX practical on modest hardware (< 5 GB extra GPU memory required for backpropagation to process a 96×96×96 volume with a four-stage network and 9 output classes). To show the feasibility of LoMiX with 3D networks, the new preliminary results of 3D MedNeXt with LoMiX are reported in Table R1 below.

Table R1: MedNeXt's performance on 3D Synapse 8-organ segmentation with Last Layer (LL), Deep Supervision (DS), MUTATION, and LoMiX. DICE scores (%) reported for Gallbladder (GB), Left kidney (KL), Right kidney (KR), Pancreas (PC), Spleen (SP), and Stomach (SM). Our results (in bold) demonstrate that LoMiX achieves the best average DICE and HD95 scores among all supervisions.

Q3: Are the W_S in Eq. (3) normalized to 1? I find it surprising that figure S.8 in the supplementals show that addition fused logit maps get weights so much higher than the concat ones, yet still are shown to be of the most important in the ablation study. Is there an explanation to why this is the case?

Response: No, $W_S$ in Eq. (3) is the $1\times1$ convolution weight matrix used inside the concat fusion; $W_S$ is a standard linear projection and it is not normalized. The bars in Fig. S8, however, depict the Softplus loss weights w from Eq. (6) for the best-epoch: independent scalar factors that rescale each loss term and are likewise unconstrained, so their magnitudes should not be interpreted as direct "importance" scores.

The apparent mismatch arises because concat’s contribution is better reflected by its complementary signal and training dynamics, not by its final scalar weight. Channel-wise concatenation injects cross‑stage information that additive/multiplicative operators may not capture, which is why ablating concat degrades DICE. Moreover, Fig. S8 shows only the best epoch; earlier curves (Supplementary Figs. S1-2, S6) reveal that concat fused logits receive higher weights during earlier training epochs, helping shape the representation before the optimizer redistributed emphasis. In short, smaller final $w$ does not imply low utility; concat remains important because of the distinct supervision it provides. We will clarify this aspect in our revised paper.

W1: It would be nice to list all of the experiments as tables.

Response: Thank you for the suggestion. All quantitative results that appear as plots in the main paper are already reported as tables in the Supplementary pdf file (i.e., Tables S3, S6-S8 correspond to Figs. 2–5). For the camera‑ready version, we will (i) add explicit cross‑references (in the main text) to these tables, and (ii) move key results into compact tables in the main paper as space permits. This will make it easier to locate the experimental results without having to search through the entire text.

W2. ...very many different papers have been written on architectural improvements of the UNet with claims of improving Dice, but few actually convince the community of improvements good enough to be worth actual implementation. I think this paper unfortunately is likely to fall into this category.

Response: LoMiX is not another UNet variant, but a new supervision strategy at training‑time. To clarify, we do not alter the encoder/decoder blocks, we do not add new stages or resolutions, hence inference remains exactly the same (i.e., no additional parameters, zero latency overhead). Our paper proposes a fundamentally new methodology, rather than a new architecture for efficient and accurate image segmentation.

Getting into more details, during training, LoMiX mixes the existing multi‑stage logit outputs to synthesize additional intermediate predictions, thus enriching supervision without introducing new resolution features. A NAS‑inspired, Softplus weighting scheme then automatically learns how much each original or fused logit should influence the overall loss. This is why our contribution is about how to supervise, not about a new architecture.

The benefit of LoMiX is most evident in limited‑data scenarios (see Fig. 2 and Supplementary Table S3), where supervision is the bottleneck: indeed, by implicitly ensembling stage predictions during training, LoMiX provides richer, more diverse signals and yields consistent DICE gains across various networks and datasets. In other words, rather than contributing with yet another UNet variant, we offer a new, general, architecture‑agnostic supervision strategy that the community can adopt with minimal code changes and no inference overhead.

W3. From what I can tell the leap from [18] doesn’t seem to technically be very high. However, the work is thorough and the results show improvement over the previous results in [18].

Response: LoMiX makes a significant advancement over MUTATION [18] by turning supervision into an automatic, learnable, and architecture‑agnostic process that enables:

• Richer fusion space for diverse logit synthesis: Indeed, instead of a single, fixed combinatorial mutation, LoMiX defines a library of operators (Add, Mult, Concat, AWF, etc.) and systematically considers all non‑empty subsets of stage logits. This yields a broad spectrum of intermediate predictions, thus capturing complementary spatial cues that cannot be captured by using simply a single operator (i.e., Add) as in [18].
• Fully automatic supervision via NAS‑inspired Softplus weighting: We replace the uniform weights (i.e., 1.0) in [18] with a Softplus weighting scheme that learns in an end‑to‑end fashion how much each original or fused logit should contribute to the loss. As such, no heuristic tuning and no per‑dataset tweaking are needed; the weighting adapts itself during training, making supervision fully automatic.
• Training‑only, plug‑and‑play module with zero inference overhead: LoMiX operates purely at the logit/loss level. We do not modify the encoder/decoder blocks, we do not add stages, we do not keep any extra heads at inference time. The deployed network remains identical: no extra parameters, no additional latency, directly mitigating any practical adoption barriers of our approach.
• Targets data‑limited scenarios: Our largest gains appear when data are limited: the scenario where the supervision quality (not the network architecture) is critical. By implicitly ensembling multi‑stage predictions during training, LoMiX delivers consistent DICE improvements across all backbones and datasets. This trend is not observed in [18].
• Rigorous analysis and robustness checks: We go beyond headline numbers: operator and subset ablations (Fig. 3 and Supplementary Table S6), weight‑dynamics curves over training (Supplementary Figs. S1-S8), cross‑architecture evaluations (Fig. 5 and Supplementary Table S8), and data‑efficiency assessments (Fig. 2 and Supplementary Table S3), all substantiate why and how LoMiX works. This breadth of experimental evidence is absent in [18].

To sum up, LoMiX introduces a new, general, learnable, and fully automatic supervision method; this is a clear technical and practical contribution beyond the fixed, single-operation mutation strategy of [18].

[18] Rahman, M.M. and Marculescu, R. Multi-scale hierarchical vision transformer with cascaded attention decoding for medical image segmentation. MIDL, 2023.