BeyondMix: Leveraging Structural Priors and Long-Range Dependencies for Domain-Invariant LiDAR Segmentation

Thank you for your thorough review and thoughtful assessment of our manuscript. We appreciate your recognition that our work, BeyondMix, successfully identifies the fundamental limitations of existing mix-based UDA methods, provides comprehensive ablation studies, and achieves state-of-the-art performance metrics.

We acknowledge your primary concerns regarding the need for further clarification of certain aspects of our methodology. In the following response, we address each of your comments systematically to provide the requested explanations and clarifications. Please feel free to use the discussion period if you have any additional questions.

---

Q1:

What are the runtime and memory overheads introduced by BeyondMix, particularly due to Mamba and the multiple auxiliary losses?

A1:

Below, we provide a comprehensive analysis of computational requirements including GPU memory utilization, per-iteration runtime, and FLOPs for both training and inference phases.

During the training stage, the incorporation of long-range modeling mechanisms inevitably introduces additional computational overhead. However, this increase remains relatively modest due to the near-linear complexity (approximately O(n)) of our Mamba-based implementation.

For inference , our method employs a standard network architecture without auxiliary modules, resulting in computational requirements comparable to baseline approaches.

As demonstrated in Table R1, BeyondMix++ exhibits identical GPU memory consumption, per-iteration runtime, and FLOPs during inference compared to baseline methods such as LaserMix and DGT-ST. Notably, our approach achieves substantial performance improvements (+10.2% over LaserMix and +3.1% over DGT-ST) while incurring only acceptable additional training overhead, highlighting the favorable efficiency-performance trade-off of our proposed framework.

Table R1: Quantitative Analysis of Computational Overhead

---

Q2:

Are you tested or considered BeyondMix under real-to-real domain shifts (e.g., SemanticKITTI → SemanticPOSS)? Would the same priors and Mamba integration apply effectively?

A2:

In accordance with established research paradigms [CosMix, PolarMix, LaserMix, DGT-ST], we construct UDA frameworks spanning synthetic-to-real scenarios (Syn→SK, Syn→SP), capitalizing on the availability of synthetic annotations. This virtual-to-real generalization framework aligns more closely with practical deployment scenarios. Furthermore, to substantiate the generalization of our methodology, we present real-to-real adaptation results (SP→SK) in Table R2. For cross-modal sensing evaluation, we establish an SK (rotating LiDAR with 64 beams)→PGT configuration, leveraging the PandarGT (PGT, a solid-state front-facing LiDAR with 150 beams) from Pandaset. As empirically demonstrated in Table R2, our approach consistently attains state-of-the-art performance metrics. This uniform superiority across synthetic-to-real, real-to-real, and cross-sensor domain adaptation paradigms corroborates the robust generalization capacity of our proposed methodology.

Table R2: Comparison Results of Other Settings

---

Q3:

Could you report per-class IoU in the ablation studies? This would help determine whether specific object categories benefit more from certain structural priors.

A3:

As requested, we provide a comprehensive per-class performance analysis from our ablation study in Table R3.

Upon incorporating $L_{permutation}$, we observe significant improvements in the "sidewalk", "road", "person", and "pole" categories. This enhancement can be attributed to the distinct characteristics of our space-filling curves. The Hilbert curve constructs a continuous path traversing 3D voxels with strong locality preservation, resulting in more comprehensive modeling of relatively small local objects such as "person" and "pole". Conversely, the Z-curve's horizontal continuity properties offer advantages in modeling planar categories like "sidewalk" and "road".

When $L_{local}$ is introduced, substantial performance gains emerge in the "car", "motorcycle", "vegetation", "road", and "building" categories. This aligns with our local consistency structural priors, where cylindrical partitioning recognizes that objects proximal to LiDAR sensors typically include "car" and "motorcycle" instances, while distant points frequently represent "vegetation". Additionally, rectangular partitioning effectively preserves local features by leveraging the observation that "building" structures typically appear at higher elevations while "road" surfaces are situated at lower positions. These experimental results corroborate our theoretical framework.

The addition of $L_{geometric}$ yields balanced improvements across all categories, indicating that geometric consistency across varying point densities serves as a universal constraint applicable to diverse semantic classes.

Table R3: Ablation Study of Per-class Performance.

---

Q4:

Could you clarify how Mamba is integrated into your architecture (e.g., at voxel level or raw sequence)? Additionally, how does your use of Mamba compare in motivation and performance to recent long-range modeling methods such as PointTransformer v3? A conceptual and empirical contrast would help clarify the choice.

A4:

We appreciate your inquiry and would like to provide a detailed explanation:

To effectively model long-range dependencies, we deploy Mamba within the feature space generated by the segmentation model's encoder. This feature space offers significant advantages: reduced spatial sizes, higher dimensionality, and semantically enriched representations. By serializing these features and processing them through the Mamba architecture, we achieve alignment between corresponding features from different sequential orderings. This approach facilitates the encoder's acquisition of structural priors, thereby enhancing segmentation performance in the target domain.

Regarding your suggestion about Point Transformer v3 (PTv3), which implements attention mechanisms within locally grouped point cloud sequences, our analysis reveals that the standard PTv3 implementation possesses a receptive field limited to 1024, which proves insufficient for our application. As demonstrated in line 71 of our manuscript, the minimum requisite receptive field exceeds 1300 (potentially considerably more, contingent upon mixing strategies or scanning methodologies). However, Mamba exhibits an unbounded receptive field, rendering it particularly suitable for modeling long-range dependencies essential to our task domain. As presented in Table R4, our theoretical assertions are substantiated by quantitative experimental results. Concurrently, the Pamba study [1] illustrates in Figure 1 that Mamba's effective receptive field substantially surpasses that of PTv3.

Furthermore, Pamba [1] establishes that "SSM can process the whole 100,000+ points expeditiously without subdivision and model uncompressed long-range dependencies," asserting that "SSM is more suitable than transformer for extracting long-term dependencies." This aligns precisely with our requirements, as each scan in our outdoor LiDAR dataset comprises over 100,000 points. While we experimented with PTv3 (P=2048), the quadratic computational complexity inherent to transformer architectures rendered the processing speed prohibitively inefficient for practical implementation.

[1] Pamba: Enhancing Global Interaction in Point Clouds via State Space Model. AAAI2025

Table R4: Comparison between Different Long-range Modeling Methods.

We greatly appreciate your time and thoughtful assessment of our manuscript. We are particularly grateful for your recognition that our BeyondMix approach effectively addresses the challenges of cross-domain feature representation in LiDAR-based applications.

We have carefully considered your primary concerns regarding your specific areas of interest and the inadvertent errors in our manuscript. In the following sections, we address each of your comments systematically to improve the clarity and accuracy of our paper. Please feel free to use the discussion period if you have any additional questions.

---

Q1:

The permutation invariant prior module only selects two specific sorting sequences and learns to make their representations in the Mamba space approach each other, rather than making their representations similar in any sorting. Therefore, whether it can be called "permutation invariant" remains to be considered.

A1:

We apologize for any confusion regarding our methodology. While numerous space-filling curve algorithms exist, we deliberately selected two of the most widely adopted implementations—H-curve and Z-curve—to generate diverse sequential representations. Our approach specifically aims to ensure that feature representations across these different scanning orders exhibit proximity in the Mamba embedding space. For clarity, we will revise our terminology to "H-Z permutation invariance" to explicitly indicate the two specific space-filling curves employed in our framework. This modification will be incorporated in subsequent versions of the manuscript to enhance precision and prevent misinterpretation.

---

Q2:

The meaning of figures 1 (b) and (c) is unclear, especially the table in (c) is difficult to understand.

A2:

We apologize for any confusion and will address this issue in the final manuscript.

In our notation, "-" indicates structural priors not previously explored in existing methods, specifically permutation invariance, local consistency, and geometric consistency. The "x" symbol denotes structural priors that fundamentally cannot be implemented through mix-based approaches.

Current mix-based methodologies each leverage a single structural prior:

• Cosmix employs Category Semantic Integrity by utilizing semantic category instances as the minimal mixing unit (illustrated in Fig. 1(b), first diagram)
• Polarmix implements Azimuthal Semantic Consistency by extracting and rotationally mixing points of identical semantic classes (depicted in Fig. 1(b), second diagram)
• Lasermix enforces Spatial Region Integrity by partitioning scenes according to inclination patterns (shown in Fig. 1(b), third diagram)

These methodologies are limited by their singular focus on one structural prior and insufficient receptive field capacity. Our approach incorporates both mix-compatible priors and those incompatible with mixing techniques. The additional 3 structural priors we adopt are:

1. Permutation Invariance (Fig. 1(b), fourth diagram): Ensures consistency across different scanning orders representing varied observational perspectives (e.g., top-down versus bottom-up)
2. Local Consistency(Fig. 1(b), fifth diagram): Leverages statistical partitioning. For instance:
   • Cylindrical partitioning recognizes proximity-based semantic patterns (objects near sensors are typically" car"or "motorcycle" while distant points often represent "vegetation")
   • Rectangular partitioning utilizes Z-axis coordinates (lower values correspond to" road" while higher values indicate "buildings")
3. Geometric Consistency (Fig. 1(b), sixth diagram): Maintains consistent geometric representations across varying LiDAR point densities.

Additionally, our method models long-range dependencies, substantially expanding the receptive field and addressing the domain-invariant feature learning limitations in unsupervised domain adaptation caused by restricted receptive fields.

---

Q3:

The Mamba model in the paper is only used as an auxiliary supervision module during the training phase. Has consideration been given to directly decoding Mamba features for classification during inference? Will this design have better results.

A3:

We acknowledge the suggestion regarding direct decoding of Mamba output features. As empirically demonstrated in Table R1, this approach induces substantial performance degradation. Our comprehensive analysis elucidates several contributing factors:

During the training phase, our architecture leverages Mamba to process intermediate-layer features, establishing structural priors and effective receptive fields that constrain our encoder toward learning more discriminative and domain-invariant representations. However, Mamba transforms these intermediate features into an alternative high-dimensional alignment space that is fundamentally incongruent with the original feature manifold.

This transformed representation space lacks decoder-specific optimization, as the decoder was not trained to process features from this alternative embedding space. Consequently, the semantic decoding process fails to effectively interpret these representations, resulting in substantial performance deterioration.

These empirical findings substantiate that our architectural design—utilizing Mamba as a constraint mechanism during training rather than as a feature extractor during inference—constitutes the optimal configuration for cross-domain feature learning in LiDAR semantic segmentation.

Talbe R1: Comparative Analysis of Decoding Features.

---

Q4:

The receptive field at point B in Figure 2 contains two types of domains, but line 76 in the text states that they are not included

A4:

We sincerely apologize for the confusion caused by our inadvertent error. Thank you for bringing this issue to our attention. We incorrectly reversed the labels of point A and point B in Lines 64 and 76 of our manuscript.

For clarification, the caption of Figure 2 contains the correct description. We will rectify this error in the subsequent version of our manuscript to ensure consistency throughout the document.

We sincerely appreciate your time and effort in reviewing our manuscript. We are particularly grateful for your recognition that our work, BeyondMix, is "highly innovative," "well-written and easy to understand," with "effective results."

We acknowledge your primary concerns regarding the missing comparisons and additional issues you highlighted. All your comments are addressed point by point in the following. Please feel free to use the discussion period if you have any additional questions.

---

Q1:

To construct the various structural priors, the method requires multiple serializations of the features and repeated forward passes through the Mamba model. This may introduce considerably higher computational complexity than baseline methods. The paper lacks a quantitative analysis of runtime evaluation (e.g., inference time and FLOPs).

A1:

We appreciate your suggestion regarding runtime evaluation. Below, we provide a comprehensive analysis of computational requirements including GPU memory utilization, per-iteration runtime, and FLOPs for both training and inference phases.

During the training stage, the incorporation of long-range modeling mechanisms inevitably introduces additional computational overhead. However, this increase remains relatively modest due to the near-linear complexity (approximately O(n)) of our Mamba-based implementation.

For inference, our method employs a standard network architecture without auxiliary modules , resulting in computational requirements comparable to baseline approaches.

As demonstrated in Table R1, BeyondMix++ exhibits identical GPU memory consumption, per-iteration runtime, and FLOPs during inference compared to baseline methods such as LaserMix and DGT-ST. Notably, our approach achieves substantial performance improvements (+10.2% over LaserMix and +3.1% over DGT-ST) while incurring only acceptable additional training overhead, highlighting the favorable efficiency-performance trade-off of our proposed framework.

Table R1: Quantitative Analysis of Computational Overhead

---

Q2:

It's common for UDA methods like DACS, DAFormer, and DGT-ST to use a combination of source loss (L s) and mixed-sample loss (L mix). If L mix is already being used to adapt to the target domain, why is the separate target loss, L t, also necessary?.

A2:

Thank you for raising this important question regarding the necessity of target loss ($L_t$) in mix-based methods. We have conducted thorough investigations into this matter.

While 2D UDA methods such as DACS and DAFormer employ loss functions formulated as $L_s+L_{mix}$, the discussion and empirical evaluation of mixing strategies in 2D domain are relatively limited. Given that our proposed approach falls within the 3D methodological framework, we primarily focus our investigation on 3D LiDAR-based techniques when examining domain adaptation strategies:

• CosMix performs mutual mixing of source and target domains without intermediate forms, utilizing a total loss function of $L_{s->t}+L_{t->s}$ as defined in Equation 5 of the original paper.
• PolarMix is primarily a data augmentation method without explicit specification regarding target loss utilization.
• LaserMix, the current state-of-the-art mixing approach, incorporates a comprehensive loss function (Equation 9 of the original paper): $L_s+L_{mix}+L_{mt}$ , where $L_{mt}$ represents the target loss. Their ablation studies in Table 3 (experiments (2) and (3) of the original paper demonstrate that removing target loss results in approximately 3% mIOU degradation.
• DGT-ST follows LaserMix's framework with a total loss function (Equation 15 of the original paper): $L_s+L_{mix}+L_{sac}$ , where $L_{sac}$ is defined in Equation 12 of the original paper as essentially a target loss, albeit with unique teacher-student network input designs. Their ablation studies in Table 5 of the original paper confirm that eliminating $L_{sac}$ causes a 1.1% mIOU performance drop.

Our experimental results in Table R2 similarly indicate performance degradation when target loss $L_t$ is removed from BeyondMix++. We posit that mix methods effectively learn cross-domain commonalities, while target loss $L_t$ enables focused learning of target domain-specific 3D knowledge, thereby enhancing model performance in the target domain.

Table R2：Comparative Analysis of Target Loss $L_t$

---

Q3:

The Mamba model captures long-range dependencies along the 1D path ("flatten" a 3D structure into a 1D sequence), but its ability to capture complex 3D spatial relationships that do not lie on this path may be weaker. This could lead to the loss of critical spatial information.

A3:

Your observation regarding the potential loss of 3D spatial relationships when flattening 3D structures into 1D sequences is valid. However, our approach serves as a complementary component to existing methods rather than a replacement. Our framework preserves the original spatial relationship learning while introducing additional structural priors that were previously underexplored.

Specifically, we model long-range dependencies that prior approaches have neglected. By employing different sequential orderings, each capturing distinct spatial adjacency patterns and structural priors, our method effectively extracts domain-invariant features at individual spatial points—a critical aspect in unsupervised domain adaptation tasks.

The diverse sequential representations enable the capture of complementary structural information while maintaining the integrity of the original spatial context, thereby enhancing the model's capacity to learn transferable features across domains without compromising spatial understanding.

---

Q4:

The method enforces "equivariance" on intermediate features by inverse-mapping the Mamba outputs back to their original order. However, the ultimate goal for segmentation is to achieve "invariance" in the final prediction.

A4:

Our approach can be elucidated through two principal perspectives:

(1) Intermediate feature equivariance facilitates terminal invariance. Numerous classical paradigms, particularly consistency regularization frameworks, leverage data augmentation operations such as mixup and cropping to induce equivariance in the feature space while imposing invariance constraints in the prediction manifold. This methodology enhances model robustness against such augmentations, enabling the acquisition of more discriminative representations and consequently improving model efficacy.

(2) Within the context of UDA for 3D segmentation tasks, an idealized model equipped with perfect structural priors would exhibit invariance to diverse structural perturbations (e.g., heterogeneous scanning methodologies). Previous approaches frequently induce equivariance to structural perturbations, primarily due to the insufficient incorporation of structural priors. We address this limitation by integrating previously neglected structural priors (including Permutation Invariance, Local Consistency, and Geometric Consistency) prior to intermediate feature representation learning. This strategic integration alleviates equivariance effects, amplifies the encoder's invariance to structural perturbations, and constructs a substantially more robust model. The resultant performance improvements are empirically substantiated in Table 3 of the main text.

---

Q5:

The proposed Permutation Invariance and Local Consistency priors all work by altering the processing order of points and then enforcing consistency on the feature representations. It seems to be be conceptually overlapping.

A5:

We would like to further clarify the distinction between these two structural priors.

Permutation Invariance leverages different space-filling curves (H-curve, Z-curve) to create diverse scanning sequences. Intuitively, this enables the modeling of long-range dependencies from different perspectives (e.g., top-down versus bottom-up approaches). This structural prior can be conceptualized as consistency across different viewpoints, ensuring feature representations remain invariant despite changes in observation sequence.

Local Consistency priors utilize statistical partitioning based on local characteristics. For instance:

• Cylindrical partitioning recognizes that objects proximal to LiDAR sensors frequently belong to categories like "car" or "motorcycle," while distant points often represent "vegetation" (as validated in the LaserMix).
• Rectangular partitioning leverages the global Z-axis coordinate as a discriminative metric, where lower Z-values typically correspond to "road" classes, while higher Z-values frequently indicate "building" structures.

By implementing various local partitioning schemes, we generate distinct sub-regions within which we apply sequential ordering. This approach effectively preserves local consistency priors and enhances the model's ability to capture domain-invariant features within contextually similar regions.

Consequently, Permutation Invariance and Local Consistency priors leverage fundamentally distinct structural information, enabling complementary feature representation learning across different organizational hierarchies of the LiDAR data.

Thank you for your question.

Our method introduces additional modules and computations, resulting in a reasonably higher computational load, as reflected by the GFLOPs (76.84 vs. 63.68). However, there appears to be a misunderstanding regarding the time metric reported in Table R1 of A1. Specifically, the "Time" column is expressed in iterations per second (iteration/s), rather than seconds per iteration (s/iteration). A higher iteration/s value indicates a faster training speed.

In our case, BeyondMix++ achieves 0.67 iteration/s, while DGT-ST achieves 0.95 iteration/s, meaning that BeyondMix++ is approximately 30% slower per iteration, which aligns with the higher GFLOPs you observed.

Please don’t hesitate to let us know if there are any further questions — we would be happy to clarify.

Best regards,

Authors of Submission 16945

We sincerely appreciate your thorough review and constructive feedback on our manuscript. We are particularly grateful for your recognition of our work as "excellent" and your acknowledgment that our experimental validation is "complete and sufficient" with "clear and comprehensible" illustrations and exposition.

We see that your main concerns are further explanation of the certain ambiguous aspects in our manuscript. All your comments are addressed point by point in the following. Please feel free to use the discussion period if you have any additional questions.

---

Q1:

Some multi-modal UDA methods for point cloud segmentation also needs to be discussed in related work.

A1:

We appreciate your valuable feedback. We have supplemented our related work section with multimodality discussions and will incorporate this content into our final manuscript.

Unsupervised domain adaptation (UDA) for 3D semantic segmentation has garnered increasing academic attention, particularly regarding the exploitation of multimodal information streams. Seminal investigations [1, 2] explored the utilization of auxiliary modalities such as depth—available exclusively during source domain training—to facilitate adaptation of 3D semantic segmentation frameworks. Subsequently, xMUDA [3] established a paradigmatic approach by enforcing cross-modal consistency constraints, thereby enabling bidirectional knowledge transfer between image and point cloud representations to enhance domain generalization capabilities. Further advancements to this paradigm [4] incorporated sophisticated cross-modal fusion mechanisms and contrastive learning objectives to optimize representation alignment across both modalities and domains. Contemporary research has extended these foundational approaches through integration of advanced vision architectures and refined fusion methodologies. For instance, [5] incorporates the Segment Anything Model (SAM) to augment 2D modality representations, consequently enhancing 3D segmentation performance through more effective cross-modal knowledge transfer. The authors of [6] propose a sequential fusion-then-distillation framework, which first aligns 2D and 3D feature representations within a shared latent manifold before employing positive distillation techniques to preserve complementary modality-specific information during the adaptation process. Furthermore, [7] introduces an adaptive regularization framework for modality-guided feature fusion, facilitating dynamic and contextually appropriate integration of visual and geometric information under domain distribution shifts.

[1] Spigan: Privileged adversarial learning from simulation. ICLR, 2019

[2] Dada: Depth-aware domain adaptation in semantic segmentation. ICCV, 2019.

[3] xMUDA: Cross-modal unsupervised domain adaptation for 3D semantic segmentation. CVPR,2020

[4]Cross-modal learning for domain adaptation in 3D semantic segmentation. TPAMI, 2022

[5]Learning to adapt sam for segmenting cross-domain point clouds. ECCV 2024

[6]Fusion-then-Distillation: Toward Cross-modal Positive Distillation for Domain Adaptive 3D Semantic Segmentation. TCSVT 2025

[7]Exploring Modality Guidance to Enhance VFM-based Feature Fusion for UDA in 3D Semantic Segmentation. CVPR 2025

---

Q2:

Figure 1(c) is confusing. What's the difference between the symbol "-" and "x"? Could the author provide a detailed explanation? It would be best for the author to provide the strategies adopted by the specific method.

A2:

We apologize for any confusion and will address this issue in the final manuscript.

In our notation, "-" denotes structural priors not previously explored in the literature, specifically permutation invariance, local consistency, and geometric consistency. The "x" symbol indicates structural priors that cannot be implemented using mix-based methodologies.

Existing mix-based approaches each leverage a single structural prior:

• Cosmix utilizes Category Semantic Integrity by employing semantic category instances as the minimal mixing unit(illustrated in Fig. 1(b), first diagram)
• Polarmix implements Azimuthal Semantic Consistency by extracting and rotationally mixing points of identical semantic classes (depicted in Fig. 1(b), second diagram)
• Lasermix enforces Spatial Region Integrity by partitioning scenes according to inclination patterns (shown in Fig. 1(b), third diagram)

These methodologies are limited by their singular focus on one structural prior and insufficient receptive field capacity. Our approach incorporates both mix-compatible priors and those incompatible with mixing techniques. The additional 3 structural priors we adopt are:

• Permutation Invariance (Fig. 1(b), fourth diagram): Ensures consistency across different scanning orders representing varied observational perspectives (e.g., top-down versus bottom-up)

• Local Consistency(Fig. 1(b), fifth diagram): Leverages statistical partitioning. For instance:

  • Cylindrical partitioning recognizes proximity-based semantic patterns (objects near sensors are typically" car"or "motorcycle" while distant points often represent "vegetation")
  • Rectangular partitioning utilizes Z-axis coordinates (lower values correspond to" road" while higher values indicate "buildings")

• Geometric Consistency (Fig. 1(b), sixth diagram): Maintains consistent geometric representations across varying LiDAR point densities.

Additionally, our method models long-range dependencies, substantially expanding the receptive field and addressing the domain-invariant feature learning limitations in unsupervised domain adaptation caused by restricted receptive fields.

---

Q3:

Line 247, BeyondMix++ randomly selects from various existing mixing strategies. Could the author present the result of each mixing strategy? And which strategy best fits the proposed BeyondMix.

A3:

As demonstrated in lines 329-336 of the main text and in Table 7, we systematically evaluated the performance gains achieved by integrating BeyondMix with previous mixing strategies (CosMix, PolarMix, LaserMix) and their combinations. Our experiments reveal that when combining BeyondMix with individual mixing methods, LaserMix achieves the highest performance (45.4%). We attribute this to LaserMix's more comprehensive structural priors and effective cross-domain receptive field construction. When pairing BeyondMix with two mixing methods simultaneously, the CosMix+LaserMix combination yields the best results (46.0%). This can be explained by the operational similarities between PolarMix and LaserMix—both involve radius-based manipulations relative to the central point, albeit through different mechanisms—making CosMix+LaserMix a more complementary pairing. The integration of all three strategies achieves optimal performance (46.2%), confirming that our BeyondMix++ successfully leverages the diverse structural priors and strengths of each mixing method to capture a more comprehensive range of domain-invariant features for robust LiDAR semantic segmentation.

Table 7: Comparison between different mix strategies.