SR$^2$: BOOSTING 3D LARGE LANGUAGE MODEL WITH SPATIAL RELATION REASONING

1. The improvement after incorporating SRR is not so significant on most metrics according to Table 1. Considering this point, I think the efficiency of SRR should be provided, e.g., additional inference time, memory footprint, which can demonstrate a comprehensive tradeoff.
2. In Table1, there is no other method reported on 3D ReasonSeg benchmark. The authors should implement some representative methods on this for fair comparison.

The experimental results indicate that the improvement brought by $SR^2$ is relatively marginal. Specifically, the performance gain is only 0.1 on ScanRefer Acc@50 and 1.5 on the 3D ReasonSeg dataset.

Minor issue: Inconsistent terminology: The $SR^2$ method is inconsistently referred to as SPR in L227 and L295.

1. The authors have set up a new benchmark and claim that the proposed new benchmark can provide a more comprehensive evaluation in terms of the 3D spatial reasoning capability of the models. It would be better if the authors can have a table to summarise the different between the proposed dataset compared previous ones to make the contributions and differences more clear.

2. As in table 1, the improvement of adding SR^2 is not significant - only about 1% for most of the metrics. It would be more convincing if more improvement is brought by the proposed pipeline.

• In my view, the proposed framework is a two-stage sequential reasoning process, where stage one detects relevant objects and stage two reasons on these sampled objects. Such a pipeline is quite straightforward, lacking some technical contributions. 

• I believe 3D spatial information such as 3D coordinate and geometry information is fundamental in distinguishing 3D tasks from 2D tasks. However, how to better leverage such 3D spatial information to improve 3D-LLM's spatial reasoning is still unexplored. 

• Fig.1 is somewhat blurry, making it difficult to distinguish the objects clearly.

• Besides the positional relationships between objects, I believe the geometric shapes and relative sizes of objects at varying scene scales are also crucial for 3D spatial reasoning, which is ignored in this work.

Based on above strength especially about the motivation, I would say however the proposed method seems to be too heavy. I like motivation that first locate the objects then infer the relationship. But I think the method looks very heavy and redundant. it seems that it does not necessarily call the heavy 3D-VLLM twice. It should be able to directly run an efficient 3D VLLM to locate the objects then leverage the localized 3D position for directly reasoning the relationship instead of using complex tokens from features.

Besides, if just look at baseline vs. baseline + SR2, the proposed method does not improve the performance significantly. I would also attribute the slight improvement to the redundant design since maybe the super-point grouping introduce more noisy information. More importantly, I found that the baseline the authors use already achieves very significant improvement compared to other methods. In that case, it seems that using better LLM and more advanced vision encoders are more important compared to the motivation of decomposition.

I would also recommend the author compared the latency for all the experimented baselines. Again, I like the motivation so I do expect that with the new proposed "two-phase paradigm", we can use more efficient models to achieve better performance instead of simply calling a heavy model twice while not improving much performance.