WaveFormer: Leveraging Wavelet Transformation for Multi-Scale Token Interactions in Hierarchical Transformers

1. Insufficient experimental evaluation: authors only evaluated this model on three CT datasets, and these datasets have same annotated organs. Thus, current experimental results cannot demonstrate the performance and generalizability of the network. Evaluations on other organs and other 3D modalities are necessary, such as spin segmentation in CT, tumor segmentation in CT, tubular structure (vessel) segmentation in CT and segmentation tasks on MR imaging (Brain Tumor segmentation on MR and ACDC dataset).
2. Limited comparison: authors only compared their model with baselines which were proposed before 2022. Medical image segmentation is a very popular topic, and various volumetric models have been proposed between 2022 and 2024, such as [3][4][5][6][7][8]. These models have achieved superior performance, and some have low computational complexity. Thus, current comparison results are totally insufficient. We suggest to compare with MedNext, UNETR++, and VSmTrans.
3. Low novelty. Authors propose to utilize the Wave Transform to aggregate features, but other modules and operations are borrowed from others' works, including window attention and global attention. The overall network architecture was proposed in UNETR [1] and Swin UNETR [2], and has also been widely used [3][4][6].
4. Performance is not promising. Authors propose the WaveFormer for computational efficiency, but this network do not have a low number of parameters and FLOPs. In Table 1, it has more Params and FLOPs than 3D UNet, SegResNet, RAP-Net, nnU-Net, and TransBTS. In Table 3, accuracy of WaveFormer was lower than other compared results, 80.9% for WaveFormer compared with 83.3% for RegionViT, 82.2% for Focal and 81.3% for Swin. Params of WaveFormer is 28.5M, much higher than DeIT (22.1M) and PVT (24.5M). These results demonstrated that the performance of WaveFormer is not promising.
5. Poor presentation. (1) figures and captions are overlapped, (2) missing figure labels reference in Line 421 and 430 (3) only qualitative comparison with UXNet (4) wrong bold results. In Table 3, 80.9% was not the best, but was bold. Same with 28.7M. Some results which are not the best one were bold. (4) lack sufficient descriptions in figure and table captions.
6. Missing significant descriptions and experiments. Authors evaluated the WaveFormer on ImageNet-1K, but they didn't provide architectures that they used. If they proposed WaveFormer for volumetric medical image segmentation, they didn't design experiments to compare it with DeIT, PVT, RegionViT, Focal, and Swin on medical image segmentation tasks. If you propose the WaveFormer as a general Transformer architecture, they didn't design experiments in semantic segmentation. We suggest to compare WaveFormer with Focal and Swin on Medical Image segmentation by incorporating them into the same segmentation architecture.

Minors. (1) Lacking training and test time. (2) Inappropriate descriptions in Abstract: compare with current SOTA transformers, but transformers compared in this manuscript are not SOTA now in 2024 (e.g. SwinUNETR and UX Net). Thus, comparing WaveFormer with them cannot demonstrate its superiority. (3) lack p-values and standard deviation (4) No visual interpretation for Wave transform.

[1] Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., ... & Xu, D. (2022). Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp. 574-584). [2] Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. R., & Xu, D. (2021, September). Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI brainlesion workshop (pp. 272-284). Cham: Springer International Publishing. [3] Liu, T., Bai, Q., Torigian, D. A., Tong, Y., & Udupa, J. K. (2024). VSmTrans: A hybrid paradigm integrating self-attention and convolution for 3D medical image segmentation. Medical Image Analysis, 98, 103295. [4] Shen, Q., Zheng, B., Li, W., Shi, X., Luo, K., Yao, Y., ... & Wei, Q. (2024). MixUNETR: A U-shaped network based on W-MSA and depth-wise convolution with channel and spatial interactions for zonal prostate segmentation in MRI. Neural Networks, 106782. [5] Roy, S., Koehler, G., Ulrich, C., Baumgartner, M., Petersen, J., Isensee, F., ... & Maier-Hein, K. H. (2023, October). Mednext: transformer-driven scaling of convnets for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 405-415). Cham: Springer Nature Switzerland. [6] Shaker, A. M., Maaz, M., Rasheed, H., Khan, S., Yang, M. H., & Khan, F. S. (2024). UNETR++: delving into efficient and accurate 3D medical image segmentation. IEEE Transactions on Medical Imaging. [7] Chen, J., Mei, J., Li, X., Lu, Y., Yu, Q., Wei, Q., ... & Zhou, Y. (2023). 3d transunet: Advancing medical image segmentation through vision transformers. arXiv preprint arXiv:2310.07781 [8] Azad, R., Niggemeier, L., Hüttemann, M., Kazerouni, A., Aghdam, E. K., Velichko, Y., ... & Merhof, D. (2024). Beyond self-attention: Deformable large kernel attention for medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1287-1297).

Missing recent state-of-the-art models during the comparison, e.g., Auto3DSeg and MedNeXt

Though the paper delivers a novel idea, it is disorganised. For example, figure references are missing in various places (Line 421, 430), and figures and captions are not appropriately aligned (Figure 4).

In short, the paper does not substantiate its claims, lacks thorough analysis to support its methodology, and suffers from a general lack of clarity.

• (W1) The title of the paper is phrased as a general methodology for both 2d and volumetric data, however the methodology is throughout build as an evolution of the UNETR architecture for volumetric medical data and all figures center around 3D input. The application to 2D data is non-trivial, and not explained including in the appendix.
• (W2) Experimental results on Imagenet-1k does not meaningfully beat prior methods, and the authors does not provide any analysis to justify the applicability of the Waveformer methodology for this data.
• (W3) The paper claims to be evaluated on the KiTS dataset (mentioned both in abstract and in section 5), however the results on this dataset are not given.
• (W4) The paper generally lacks in-depth analysis of the effects and implications of performing attention on DWT coefficients. In particular, the claims made on lines 205-215 would be very relevant for empirical analysis, however this is missing from the paper.
• (W5) While the authors claim that the performance boost of the methodology is caused by enlarged per layer receptive field, the two MICCAI challenge datasets used are generally characterised by requiring only local information (e.g. intra organ features) to perform, and the authors provide no explanation for this.
• (W6) In general, the paper lacks clarity. The writing is overly complicated and the figures of the paper are generally neither well-made nor easily digestible. In particular Figure 2, which is of major importance to understand the paper, is difficult to understand. Further, Figure 3 is almost a copy of the figure provided in SwinUNETR [1]. Lastly, the figure text is on top of Figure 4.
• (W7) Paper lacks central details on the DWT implementation, e.g. choice of kernel.

1. Some references are missing (e.g., Lines 421 and 430).

2. Captions and figures overlap (e.g., Line 403).

3. The overall improvement in segmentation accuracy is marginal, with roughly a 1% Dice increase over SwinUNETR on the FLARE 2021 dataset, while computational complexity remains similar. A statistical significance test is recommended to confirm if the improvement is meaningful.

4. GPU memory footprint comparisons are absent; reductions in FLOPs and parameters alone appear marginal.

5. The paper appears incomplete.