PT-T2I/V: An Efficient Proxy-Tokenized Diffusion Transformer for Text-to-Image/Video-Task

W3: The Extraction of Proxy Tokens

The extraction of proxy tokens through averaging is straightforward, but how representative are these proxy tokens when obtained in this manner? Could proxy tokens also be extracted using clustering or similarity-based methods for potentially improved representativeness?

A3: Thank you for your suggestion. Extracting proxy tokens based on similarity may introduce additional computation, which could reduce the efficiency of PT-DiT. We conduct ablation experiments to explore the impact of similarity-based proxy token selection. Specifically, during each sampling, the cosine similarity of each token with all other tokens in the window is calculated after L2 normalization, and the token with the highest sum of similarities is chosen as the proxy token. The results, shown below, indicate that while the similarity-based approach provides only a modest performance improvement, it also introduces greater computational complexity. To strike a better balance between computational efficiency and performance, the average-based approach remains the more suitable option.

Q1: What accounts for PT-DiT's 80% improvement over Lumina in Figure 8 (left), especially since this enhancement is significantly greater than the improvements observed for other models?

A4: Thank you for your question. We believe this difference is due to the specific design of the Lumina-Next structure. With a higher number of transformer blocks (1152, compared to just over 30 in PT-DiT/H), Lumina-Next includes more attention operations. As a result, PT-DiT, which is based on the proxy token mechanism, can offer more significant advantages in terms of efficiency.

Q2: How about the training time and inference time of PT-DiT?

A5: Thank you for your question. We test the training and inference times of PT-DiT/S-Class, and as expected, our method successfully accelerates both model training and inference, as demonstrated by the following results.

Q3: Does the author have any plans to open source the Qihoo-T2X series of models and code?

A6: Thank you for your interest. We are open-sourcing the entire Qihoo-T2X family of models and code to support the development of the efficient diffusion transformer.

General Reply

Thank you very much for your valuable comments. Below, we provide detailed responses to each of your questions and comments. If you have any further questions or concerns regarding our paper or our responses, please let us know, and we will address them promptly.

W1: A Fair Comparison on ImageNet

This article lacks results on ImageNet for a fair comparison with existing state-of-the-art diffusion transformer models. Previous studies, such as DiT, MDT, VAR, and SiT, have all been evaluated on ImageNet using metrics like FID and IS. To fully establish its effectiveness, PT-DiT should also be assessed under similar conditions. This is essential for demonstrating its competitive performance despite reduced computational complexity and should be included in evaluations.

A1: Thanks for your valuable suggestion. Following your advice, we conduct experiments on ImageNet at a resolution of 256, aligning PT-DiT/L-Class (693M) with DiT-XL (675M), and ensuring consistent training and inference setups for a fair comparison. Due to time constraints, we train for 2,000,000 steps. For evaluation, we randomly sample 50,000 images for testing and assessed performance using metrics such as FID and IS. The results, shown below, demonstrate that, thanks to the well-designed agent token mechanism and TCM, our method achieves competitive performance with DiT-XL while offering a clear efficiency advantage.

W2: The Value of Compression Ratio

How is the hyper-parameter compression ratio determined in this paper? Why is the compressed ratios set to (1, 2, 2) at a resolution of 256, and will a ratio of (1, 16, 16) result in excessive information loss at a resolution of 1024? The authors should provide further clarification on the rationale and motivations behind the chosen compressed ratios settings in their methodology.

A2: Thank you for your insightful suggestion.

We first determine the compression ratio at a resolution of 256. At this resolution, after applying the VAE (8x down-sampling) and patch embedding (2x down-sampling), the image is reduced to only 16x16 tokens. With a compression ratio of (1, 4, 4), the number of windows in the space becomes 4x4, and the number of proxy tokens is 16. We found that, with this configuration, the limited number of windows and the large coverage area of each window make it challenging for a single proxy token to effectively represent the complex information within the window for global information modeling. This leads to anomalies in the image layout and performance degradation, as shown in the Table. 2(d) in the paper. Setting the compression ratio to (1, 2, 2) maintains a reasonable number of windows while preserving the necessary semantic richness within each window. Through experimentation, we found that this setting strikes a balance between maintaining performance and improving efficiency. In contrast, the (1, 1, 1) setting does not support proxy token compression or window-attention, which are crucial for reducing computational complexity. As a result, we chose a compression ratio of (1, 2, 2) for the 256 resolution.

For higher resolutions, we found that maintaining the same number of windows across different resolutions benefits the model's training process, as it ensures a consistent semantic hierarchy across resolutions.

By continuously preserving the detail tokens, a higher compression fraction does not result in a loss of information. Following your suggestion, we will revise this section in the paper for clarity and highlight the changes in red.

Q5: The performance of T2V-MV

It's hard to evaluate the performance of T2V-MV. Can the T2V-MV work for out-of-distribution prompts? And how about the visual and quantitative evaluations with other models, given that the task setting is following previous approaches?

A6: Thank you for your question. Previous work, VideoMV, uses the video output from the text-to-multiview (T2MV) task to assist in optimizing 3D Gaussian Splatting (GS) or Neural Radiance Fields (NeRF), and then renders new views based on the learned 3D GS or NeRF, evaluating the results using PSNR, SSIM, and LPIPS metrics. Since our work focuses solely on the video generation component, a direct comparison with these previous methods is not possible. However, we provide a visual comparison between the results generated in the first stage of VideoMV and those produced by our method in an anonymous repository.

Q6: The page limit is 10, no need to move related work to the appendix.

A7: Thank you for your suggestion. Following your advice, we will incorporate the related work into the text.

Q7: A Higher Compression Ratio at 3D VAE and Patch Embedding

Can we use a VAE with a higher compression ratio, or a large patch embedding to achieve a similar level of efficiency? In principle, with the proposed design, more details could be restored compared with directly using a higher compression ratio (but with vanilla attention). It would also be great to show that.

A8: Thank you for your insightful suggestions. Using larger compression ratios and larger patch embedding sizes would lead to significant performance degradation due to the loss of too much image detail. Previous experiments and studies on Stable Diffusion and Diffusion Transformers have validated 8x spatial compression and 2x patch embedding compression as more efficient settings that avoid significant performance loss. Therefore, following previous works, our approach does not increase this compression ratio, instead maintaining the token count to preserve the richness of detailed information and prevent performance degradation.

Details Of Ethics Concerns: The model name "Qihoo-T2X" has nothing to do with the proposed PT-DIT but potentially indicates the identity of the researchers. The authors also did not explain the model name in the paper.

A9: Thanks to your suggestion, we will modify the title and paper of Qihoo-T2X to PT-T2I/V to match our proposed PT-DiT.

General Reply

Thank you very much for your valuable comments. Below, we provide detailed responses to each of your questions and comments. If you have any further questions or concerns regarding our paper or our responses, please let us know, and we will address them promptly.

W1 and Q3: Window Attention

W1: The method section is not clear enough. The mechanisms of window attention and shifting window attention are still not clear.

Q3: Details for the window attention and shifting window attention are missing. Only Figure 4 shows some fundamental ideas of them but how do they work exactly? How does the shifting window work?

A1: Thank you for your helpful suggestions. We will add explanations about Window Attention and Shift Window Attention in the paper and mark them in red for better understanding of the readers. Window attention and shifted window attention, proposed in Swin Transformer [1], introduce a visual inductive bias—similar to the concept of convolution—into the transformer architecture, resulting in a significant performance boost in computer vision tasks. Specifically, window attention computes attention only on tokens within a local spatial window of the image, while shifted window attention applies spatial translation to the window divisions, enabling connections between neighboring tokens across different windows. Since it is not the main contribution of this paper, more details about window attention and shifting window attention can be found in Swin-Transformer [1] and its source code (https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer.py).

[1] Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. ICCV 2021 best paper.

W2 and Q4: The Application of Text-to-MV Task

W2: The task of T2MV is a little strange. What would be an application scenario for that task?

Q4: Why Text-to-MV task is interesting? At the training level, there is no difference from this task to Text-to-video unless the training data; while for real practice, it's hard to control the generated content from only texts without any reference image.

A2: Thanks for your valuable comment. Text-to-MultiView (T2MV) has a wide range of application scenarios such as Virtual Reality (VR), E-commerce, Virtual Shopping, Game Character Design and Fashion Design. Although the output form is the same, the text-to-video task focuses on ensuring motion continuity, while the text-to-multiview task prioritizes the structural and textural consistency of the subject across multiple viewpoints. Additionally, some methods use video generation results as an intermediate step to optimize 3D Gaussian splatting, aiming to improve overall performance.

W3: The "text-to-any-task" is overclaimed. The focused three tasks are essentially video generation (with a single frame or multi-view video).

A3: Thank you for your insightful suggestion. Following your recommendation, we will revise 'text-to-any-task' to 'text-to-image/video-task.' We will explore voice and text-based tasks by adapting the 1D proxy token and 1D window attention in the future.

Q1: l201-203, f, h, w are not the actual frame, height, and width of the video/image, but the latent, right? Also, for images, how does the 3D VAE, with temporal downampling layers, handle them?

A4: Thanks for your question. Yes, in lines 201-203, f, h, w are the frame, height, and width of latent code at latent space. Current 3D VAEs, such as 3D VAE in Open-Sora, OD-VAE in Open-Sora-PlanV1.2, and MagViT in EasyAnimate, all support single-image inputs by bypassing the temporal down-sampling layers. When processing a video, the temporal down-sampling layers are applied as usual.

Q2: the texts from l242 to l244 are confusing, (pf⋅pf,ph) should be (pf, ph, pw)? so we have ph=pw = <some value depends on resolution> and pf = 1 for image and pf = 4 for videos, right?

A5: Thank you for your careful review, and we apologize for the ambiguity caused by our clerical error. Indeed, the text from lines 242 to 244, which currently refers to (pf⋅pf, ph), should be corrected to (pf, ph, pw). We set pf = 1 for images and pf = 4 for videos. We will revise the description in the paper and highlight the changes in red.

Dear Reviewer wrGZ,

Could you kindly review the rebuttal thoroughly and let us know whether the authors have adequately addressed the issues raised or if you have any further questions.

Best,

AC of Submission715

W4: The Statistical Comparison between w and w/o PT-DiT

A statistical comparison between Qihoo-T2V with and without PT-DiT is necessary for clearer insights into the model's impact.

A4: Thanks for your question. We provide a statistical comparison between Qihoo-T2V with and without PT-DiT on UCF-101 and MSR-VTT, as shown in the table below. Due to limited training time, we train the 256×256×16 video generation model on WebVid for 40,000 steps with a batch size of 256. Specifically, we use two commonly adopted video generation architectures: 2D+1D attention and 3D attention. The experimental results show that thanks to the proxy token mechanism and TCM, PT-DiT achieves performance comparable to 3D full attention, with faster convergence due to the introduction of visual priors. In contrast, the 2D+1D approach experiences more significant performance degradation.

UCF-101

MSR-VTT

W2: Further discussion

Need further discussion on the relation and differences to previous methods

A2: Thank you for your valuable suggestions, we further discuss the similarities and differences between our approach and DAM below.

Similarities: Both PT-DiT and DAM address self-attention redundancy and enhance the efficiency of the diffusion transformer by reducing the number of tokens involved in global interactions.

Differences: This paper differs from DAM in the following aspects:

1.The analysis of redundancy: We conducted a detailed analysis of the causes and distribution of redundant computations in global self-attention. Visualizing the attention map reveals that redundancy primarily arises from the sparse nature of visual information. Redundant computations mostly occur in the attention interactions between spatially neighboring tokens and distant tokens, while attention between spatially proximate tokens is critical and should not be overlooked.

2.Proxy token mechanism: We propose a spatial and temporal prior-based proxy token extraction and modeling, which is a better fit for the visual generation task. DAM may be able to implicitly achieve similar goals by proceeding with training, but it increases the burden of model learning.

3.Texture complement module: The interaction between spatially neighboring tokens is preserved through window attention and shifted window attention. In contrast, the mediator token mechanism in DAM may lose some of the spatial relationships between neighboring tokens.

4.Generic text-to-image/video task: We focus on building a powerful and foundational text-to-image and video generation model from scratch, whereas DAM is explored solely on the class-to-image task on ImageNet.

To summarize, this paper differs significantly from DAM in terms of redundancy analysis, the proposed methodology, and the primary tasks. We will add a brief discussion in the related work of the revised paper and mark it in red.

Additionally, PT-DiT doesn't analyze the significance of the observed phenomenon across different denoising timesteps, as DAM does.

Answer: Thank you for your insightful suggestions. We believe that designing dynamic compression ratio strategies for PT-DiT, determined by the observed phenomenon across different denoising timesteps, is a valuable research direction. Since each step tends to prioritize different aspects of the generated image, earlier steps may focus on building the overall structure, requiring smaller compression ratios to maintain high-quality global modeling. In contrast, later steps often focus on refining texture details and may benefit from larger window sizes and higher compression ratios for more detailed local modeling. However, adjusting the compression ratio at different steps could potentially lead to training instability, which would require the development of specialized strategies and thorough experimental validation. As a result, we did not analyze this phenomenon across different denoising timesteps or design a dynamic compression ratio strategy in the current paper. However, we plan to explore this idea in future work. Thank you again for your valuable suggestions.

W3: The effectiveness of GIIM and TCM

From the ablation study in Table 2(a), the results show that without TCM (an auxiliary module), the performance drops significantly, whereas without GIIM, which is the core component proposed in the paper, the performance decline is much less. Does this imply that auxiliary module TCM is more important than core module GIIM?

A3: Thank you for your question.
TCM is also a core module in our method. As we pointed out in Section 3.1, the association of spatially neighboring tokens is crucial and cannot be ignored in both visual understanding and generation tasks. While GIIM efficiently achieves global modeling, it cannot model detailed information in images or videos. Since generative tasks are dense prediction tasks, the absence of dense spatial proximity token associations leads to significant performance degradation.

Regarding the observation that the performance degradation of GIIM is less pronounced than that of TCM, we believe there are two main reasons:

1.GIIM focuses on constructing the overall image layout and ensuring global consistency, a property that is less sensitive to quantitative measures like FID (Fréchet Inception Distance) when compared to the more detailed characteristics captured by TCM.

2.SWA (Shifted Window Attention) indirectly facilitates the transfer of information across all tokens as the network deepens, thereby supplementing some of the global modeling capabilities.

In summary, GIIM and TCM are not independent modules, but complementary components that work together. Both are essential to the design of PT-DiT, and each plays an irreplaceable role in its effectiveness.

General Reply

Thank you very much for your valuable comments. Below, we provide detailed responses to each of your questions and comments. If you have any further questions or concerns regarding our paper or our responses, please let us know, and we will address them promptly.

W1 and Q1: Performance Comparison with DAM

Insufficent performance comparison to peer methods such as DAM [1] and AgentAttention [2]. While PT-DiT significantly outperforms baselines, its performance relative to peer methods like DAM and AgentAttention is not clearly superior (see Figure 8). Need more comparison.

A1: Thank you for your valuable suggestions. Due to an oversight in our comparison with DiT and DAM in Figure 8, we initially used the text-based image generation model for the computational complexity comparison. However, when the same model configuration (e.g., hidden dimensions, number of layers, and attention heads) is applied to class-conditioned image generation, the number of parameters in the test model increases to 192M (due to the substitution of AdaLN-Single for AdaLN), which makes the comparison unfair. To ensure a fair comparison, we have reset the parameter count of PT-DiT/B-Class to 152M, aligning it with the 131M parameters of DAM-B and DiT-B. The results, shown in the following table, demonstrate that our method offers a significant computational complexity advantage over both DiT and DAM. We will update Fig. 8 in the revised pdf.

To further compare the performance with DAM, we conducted class-to-image experiments on ImageNet 256. Since DAM is trained on a SiT-based codebase and our training time is limited, we chose PT-DiT/S-Class for a fair comparison with DiT-S, SiT-S, and DAM-S, using two different codebases (DiT-based and SiT-based). The experimental results, shown below, demonstrate that our approach significantly outperforms both DiT-S and SiT-S. This advantage is primarily due to the introduction of window-attention, which aligns with visual priors, as well as the efficiency gains from the proxy token mechanism. For DAM, PT-DiT achieves comparable performance while offering a significant computational complexity advantage (i.e., 3.8 GFLOPs for PT-DiT vs. 5.8 GFLOPs for DAM).

Dear Reviewer g2Xu,

Could you kindly review the rebuttal thoroughly and let us know whether the authors have adequately addressed the issues raised or if you have any further questions.

Best,

AC of Submission715

W3 and Q2: Novelty of PT-DiT

W3: The proposed method integrates a Swin-based architecture within the DiT-based model, employing a straightforward average operation to represent tokens. From the perspective of novelty, this contribution appears to be incremental.

Q2: Several prior works, such as ToMe, ToMeSD, TRIPS, and CrossGET, have also addressed token merging. The absence of a detailed discussion or comparison with these existing studies undermines the claimed novelty of Qihoo-T2X. I suggest that the authors more explicitly delineate the unique contributions and advantages of their approach in relation to these established methods.

A3: Thank you for your suggestion.

First and foremost, the motivation behind our research differs significantly from existing approaches. Our goal is to design an efficient Diffusion Transformer that addresses the challenges of full-attention computational complexity in high-resolution and long-duration text-to-image and text-to-video tasks. In doing so, we aim to develop a series of powerful, generalized text-to-image and text-to-video generators. Among existing approaches, ToMe, TRIPS, and CrossGET are focused on visual and textual understanding, enhancing the models' ability to process both images and language efficiently through token merging. ToMeSD, while applied to generative tasks, merely introduces token merging as an acceleration strategy for existing models.

Second, our approach differs significantly from existing token merging methods. Simple token merging faces several limitations in generative tasks and cannot effectively build a powerful visual generation model.

1. Token merging typically results in the loss of detailed information, whereas generative tasks—particularly those involving sense prediction—place more stringent demands on detail preservation. Excessive loss of detail can lead to significant performance degradation.

2. Token merging compromises the ability to model spatially neighboring tokens. By visualizing the attention map, we observe that while long-distance modeling of spatially distant tokens can introduce redundancy, the attention between spatially neighboring tokens is crucial. The token merging strategy, by fusing neighboring tokens, loses part of this spatial relationship, leading to performance degradation.

Based on the above analysis, we do not adopt the token merging strategy. Instead, we design a proxy token mechanism to establish global associations using a limited number of proxy tokens, guided by visual-spatial priors. Meanwhile, all tokens are retained to prevent the loss of detailed information, and the global associations in the proxy tokens are propagated to all detailed tokens. Additionally, the modeling of spatially neighboring tokens is enhanced through the use of window attention and shifted window attention strategies.

Extensive experimental results demonstrate that our approach can build a general and powerful image and video generation model without relying on any pre-training from existing diffusion models. It achieves competitive performance compared to current state-of-the-art image and video generation models, with notable advantages in efficiency and memory usage.

In summary, the unique contributions of our approach can be summarized as follows:

1.We conduct an in-depth analysis of the redundant computation in self-attention within Diffusion Transformers, caused by visual sparsity and redundancy. We find that redundant computations primarily occur in the attention interactions between tokens within the same spatial window and those that are spatially distant. In contrast, modeling the relationships between spatially adjacent tokens is crucial and should be preserved.

2.We designed a proxy token mechanism that leverages spatial priors to perform local token fusion and extract proxy tokens while retaining all detail tokens. Proxy token attention is then used to enable global modeling, which is propagated to all dense detail tokens, efficiently establishing global associations without losing local texture details. Additionally, window attention and shifted window attention are introduced to enhance the modeling of spatially adjacent tokens, further improving the capture of spatial proximity details.

3.Through extensive qualitative and quantitative experiments, we demonstrate that an efficient proxy-token-based diffusion model can achieve competitive performance with existing state-of-the-art text-to-image (T2I) and text-to-video (T2V) models. Our approach builds foundational T2I and T2V models for a range of generalized scenarios without relying on any pre-trained diffusion transformers, offering advantages in computational complexity.

4.We will open-source our image and video generation models to support the advancement of the efficient diffusion transformer community.

Following your suggestion, we summarize the contributions of PT-DiT in the revised PDF and highlight them in red.

General Reply

Thank you very much for your valuable comments. Below, we provide detailed responses to each of your questions and comments. If you have any further questions or concerns regarding our paper or our responses, please let us know, and we will address them promptly.

W1 and Q1: Comprehensive Evaluation of Performance and Efficiency

W1: The lack of ablation studies assessing performance metrics, such as FID with respect to GFLOPs, diminishes the practical relevance of the proposed methodology.

Q1:Although the authors provide extensive comparisons between resolution and computational metrics (e.g., memory usage, GFLOPs), I believe that additional comparisons between performance and computational cost relative to other approaches in text-to-image and text-to-multiview tasks are necessary. Assessing the trade-off between performance and computational efficiency, rather than focusing solely on resolution, would offer a more comprehensive evaluation of the proposed method's effectiveness. Since identical resolutions do not inherently guarantee equivalent performance, it is conceivable that approximate methods may result in some performance degradation.

A1: Thank you for your valuable comment. Due to differences in training data, parameter scale, and training iteration across different text-to-image and text-to-multiview methods, it is challenging to make a perfectly fair comparison of efficiency and performance for text-to-image tasks. As a result, we adopt the widely used class-to-image task on ImageNet 256 to fairly compare PT-DiT with DiT, using the same parameters, training data, and experimental configurations. Specifically, we design PT-DiT/L-Class and PT-DiT/S-Class to align with DiT-XL and DiT-S, respectively, based on parameter scale, and conduct experiments using the same training and testing frameworks as DiT. The detailed model settings can be found in Table 4 of the revised paper. The results are shown in the table below, with the data reported without the use of CFG. Compared to DiT, PT-DiT achieves competitive performance and a significant efficiency advantage with the same or fewer steps, thanks to the well-designed proxy token mechanism and texture refinement module. For example, PT-DiT/L-Class at 2000k steps (limited by time constraints) achieved an FID of 9.40, which is lower than the 9.62 FID obtained by DiT-XL at 7000k steps. Similarly, PT-DiT/S-Class at 400k steps achieved an FID of 65.86, outperforming DiT-S, which had an FID of 68.40 at the same step count. Meanwhile, the computational complexity is reduced by 30% (82.64 vs. 118.69 GFLOPs) and 37% (3.86 vs. 6.06 GFLOPs), respectively. These results demonstrate that our method can maintain competitive performance while benefiting from reduced computational complexity.

Unlike previous token merging approaches, our method preserves all detailed tokens without information loss, while also maintaining the modeling of spatially neighboring tokens—an essential aspect in visual tasks—through Window Attention. As a result, PT-DiT remains competitive and highly efficient, without suffering from noticeable performance degradation.

W2: Literature Review

The discussion and comparison do not sufficiently address related works on token merging (e.g., ToMe [1], ToMeSD [2], CrossGET [3], and TRIPS [4]), which limits the comprehensiveness of the literature review.

A2: Thank you for your suggestion. We will add the relevant literature in the related work and highlight the changes in the paper.

General Reply

Thank you very much for your valuable comments. Below, we provide detailed responses to each of your questions and comments. If you have any further questions or concerns regarding our paper or our responses, please let us know, and we will address them promptly.

W1: High Resolution Results

Insufficient experiments and lack of comparative experiments at high resolution.

A1: Thanks for your valuable suggestion. We apologize for the omission of the image resolution in the text-to-image section of the Qualitative Analysis (Sec. 4.2). To clarify, Fig. 5 presents the results of a qualitative comparison with an existing state-of-the-art model at a high resolution of 1024px, where our method demonstrates competitive performance. We will revise the paper to include this information, which will be highlighted in red. In addition, higher-resolution results (1024px * 2048px, 2048px * 1024px, 1792px * 1792px) are provided at the Anonymous Repository.

W2: The Value of Compression Ratio

How is the value of compression ratio determined, and is it related to both semantic information and resolution size.

A2: Thank you for your suggestion. We first determine the compression ratio at a resolution of 256. At this resolution, after applying the VAE (8x down-sampling) and patch embedding (2x down-sampling), the image is reduced to only 16x16 tokens. With a compression ratio of (1, 4, 4), the number of windows in the space becomes 4x4, and the number of proxy tokens is 16. We found that, with this configuration, the limited number of windows and the large coverage area of each window make it challenging for a single proxy token to effectively represent the complex information within the window for global information modeling. This leads to anomalies in the image layout and performance degradation, as shown in the Table. 2(d) in the paper. Setting the compression ratio to (1, 2, 2) maintains a reasonable number of windows while preserving the necessary semantic richness within each window. Through experimentation, we found that this setting strikes a balance between maintaining performance and improving efficiency. In contrast, the (1, 1, 1) setting does not support proxy token compression or window-attention, which are crucial for reducing computational complexity. As a result, we chose a compression ratio of (1, 2, 2) for the 256 resolution. For higher resolutions, we found that maintaining the same number of windows across different resolutions benefits the model's training process, as it ensures a consistent semantic hierarchy across resolutions. We will revise the paper to clarify this section and highlight the changes in red.

Dear Reviewer vCtb,

Kindly review the rebuttal thoroughly and let us know whether the authors have adequately addressed the issues raised or if you have any further questions.

Best, AC of Submission715