Space-Correlated Transformer: Jointly Explore the Matching and Motion Clues in 3D Single Object Tracking

1. The authors could discuss and compare their method with recent 2024 works, such as:

• Xu, A., Nie, J., He, Z. and Lv, X., 2024. TM2B: Transformer-Based Motion-to-Box Network for 3D Single Object Tracking on Point Clouds. IEEE Robotics and Automation Letters.
• Sun, S., Wang, C., Liu, X., Shi, C., Ding, Y. and Xi, G., 2024. Spatio-Temporal Bi-directional Cross-frame Memory for Distractor Filtering Point Cloud Single Object Tracking. arXiv preprint arXiv:2403.15831.
• Luo, Z., Zhang, G., Zhou, C., Wu, Z., Tao, Q., Lu, L. and Lu, S., 2024, March. Modeling continuous motion for 3d point cloud object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 5, pp. 4026-4034).

2. In the abstract, the authors claim a high speed of 60 FPS; comparing inference speeds with other methods in the experiments section would improve clarity.

3. It would be better to include the final performance in Table 4 for better comparison.

4. The authors discuss data augmentation in the experiments and ablation studies but do not mention it in the introduction or methods. It would be beneficial to elaborate on this in the methods section if it is considered an important contribution.

• The paper fails to articulate the significance of SOT compared to multi-object tracking (MOT) in the context of autonomous driving. A clear discussion on the unique challenges and benefits of SOT is missing, which is critical for understanding the practical implications of the work.
• The NuScenes dataset includes seven categories, yet the paper only reports results for five. The omission of two categories is not justified, and it raises questions about the comprehensiveness of the evaluation. This gap undermines the credibility of the performance claims.

1. Using spatial BEV representation is very common in the field of 3D LiDAR detection. Using such representation in the SOT field can not been considered as a significant contribution. The authors need to explain in their rebuttal whether there are any previous works in SOT that have used similar representations.
2. In the presentation, the authors used many formulas and complex concepts to describe some commonly known representations. For example, voxel partition, sampling, and encoding are actually very common voxelization [1] processes in the 3D detection field. Additionally, the notations are somewhat complex and lengthy, such as in Equation 3 and Equation 5.
3. The authors should include the definition of default regions in the main text, otherwise readers cannot understand why the entire SOT pipeline lacks the involvement of 3D bounding box B_(i-1).

[1] End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds. CoRL 2019.

1. Clarity in Figure 3: Figure 3 is somewhat confusing. Specifically, it is unclear how, after the squeeze operation, two tensors with shapes $C \times H \times W$, $C \times H \times W$ are transformed into a $2C \times D \times H \times W$ tensor. Further clarification on the origin and role of $D$ in this transformation would be beneficial.

2. Figure 4 illustrates the modeling of motion cues, but the motivation for using varied sizes to enhance motion clue modeling is not sufficiently explained. Additionally, it remains unclear how these varied sizes are utilized in subsequent layers. It appears there may be a typo in Equation (5), as the authors describe using varied-size regions, yet the equation employs ${VF}_{R}^{default}$. Further clarification on this point would improve the reader's understanding.

3. Explanation of Performance on Waymo Dataset: Although the authors achieve state-of-the-art performance on the KITTI and nuScenes datasets, SCTrack does not outperform MBPTrack on the Waymo dataset. The authors should provide an explanation for this discrepancy to offer insights into the model’s generalizability and performance variations across datasets.

4. Missing Citations: Several recent works that are relevant to the proposed approach appear to be missing citations:

   • (TPAMI2024) Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer

   Including these citations would strengthen the contextualization of the proposed work within the existing literature.

• Incorrect comparison in Table 1: The "Mean" performance of the proposed SCTrack is calculated by averaging results across four categories. In contrast, other methods such as MBPTrack use sample-wise average results. This inconsistency leads to an unfair comparison. Please carefully verify and ensure consistency in all reported results in your paper.
• The paper proposes a joint matching and motion modeling tracking paradigm, which jointly explores appearance and matching information. However, this maybe a bit similar to the current Spatial-Temporal transformer based approaches like MBPTrack and StreamTrack [1]. StreamTrack performs both appearance and motion modeling via the Spatial-Temporal Relation matching, which can be regarded as a combination of previous appearance and motion paradigms; Please illustrate more differences with these approaches.
• Online tracking settings (i.e., evaluation) are unclear. During the online tracking stage, does SCTrack only uses the first frame template or maintain the updated templates?
• Some Minor issues: better to call ''SCTrack'' instead of ''SCtrack'', which looks more formally; Increase the spacing between the captions in Table 5, Table 6 and Fig. 5 for improved readability.

[1] Modeling Continuous Motion for 3D Point Cloud Object Tracking;