What We Miss Matters: Learning from the Overlooked in Point Cloud Transformers

We thank the reviewer for the positive evaluation and the valuable feedback. We believe that we have fully addressed your concerns and questions, and will incorporate all points mentioned below in the final version. We would be happy to address any further questions you might have.

---

Weakness 1 & Question 3: Detailed cost analysis.

We appreciate the reviewer’s concerns regarding computational cost and have conducted a more detailed analysis.

(1) Learned Attention Initialization. Acquiring learned attention indeed requires additional training (Appendix Line 524). However, for fairness, we apply the same training configuration to the baseline. The 600-epoch results in Table 7 all use learned attention initialization (The first 300 rounds are used for initial attention learning). Regardless of whether the attention initialization is random or learned, BlindFormer consistently outperforms the baseline under identical training schedules, demonstrating that the gains stem from blindspot mining rather than longer training.

(2) VRAM usage. Using gpustat to monitor GPU memory at the same batch size (bs=32), BlindFormer occupies 11,119 MB, compared to 11,043 MB for Point-MAE, a marginal increase of about 0.7%. This includes all runtime allocations (model parameters, optimizer states, activation buffers, and CUDA memory fragmentation), not just model weights.

(3) Training time. Due to the dual-branch design, BlindFormer incurs approximately 2× training overhead. For instance, on OBJ-BG classification training, one epoch takes 43s versus 22s for Point-MAE. Crucially, this additional cost is confined to training and does not affect inference speed, making the method practical for deployment. For a fairer comparison, we explain this in detail in the next weakness.

---

Weakness 2 & Question 1: Fair comparison of data samples.

We thank the reviewer for raising this concern. To further clarify, we conduct additional experiments on Point-MAE and PointGPT-S with double batch size under the same training schedule, as shown in Table A. The results on OBJ-BG show only marginal gains (+0.2% on Point-MAE and +0.3% on PointGPT-S), below the improvements obtained by BlindFormer (+0.9% and +0.7%, respectively). Similar results are observed on OBJ-ONLY. This confirms that BlindFormer’s performance boost arises from its blindspot mining and feature alignment mechanisms, rather than increased batch size. Moreover, BlindFormer’s inference cost remains identical to the baseline, as the blindspot branch is only used in training.

Table A. Performance with the same data sample size.

---

Weakness 3 & Question 4: Design choice of perturbation probability.

We appreciate your simplification of our formula and would like to clarify the role of the perturbation term.

(1) Derivation of probability distribution. Your series of derivations are reasonable. Please allow us to continue the derivation from the perspective of probability. For a random number $\epsilon \sim U[0, 1]$, the cumulative distribution function $F_Y(y)=P(-log\epsilon \le y) = P(\epsilon \ge e^{-y}) = 1- P(\epsilon \le e^{-y}) = 1- e^{-y}$. In other words, although the range of $-log\epsilon$ is theoretically $(0, +\infty)$, it is highly concentrated. (e.g. $P(-log\epsilon \le 3)\approx0.95$, $P(-log\epsilon \le 7)\approx0.999$). In practice, the attention score is still the dominant variable, and the perturbation only slightly alters mask selection to promote diversity.

(2) Additional theoretical explanation. The perturbation term $-log(-log\epsilon)$ is called Gumbel noise, which is a random variable sampled from the Gumbel distribution. It is widely used in reinforcement learning and generative models to introduce randomness. Given our core contribution is not an improvement based on this, more detailed mathematical theory and related work can be referred to [1, 2]. Thank you again for your concern. We will add relevant literature to the final version to facilitate readers' understanding.

[1] Maddison, et al. "The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables." ICLR, 2017.

[2] Oberst, Michael, and David Sontag. "Counterfactual off-policy evaluation with gumbel-max structural causal models." ICML, 2019.

---

Weakness 4: Design inspiration from the ViT community.

We thank the reviewer for their valuable suggestions. As you mentioned, in the ViT community, adding registration markers has been shown to mitigate the loss of geometric discriminability. BlindFormer approaches this issue from a different perspective, encouraging the model to explore blindspots in sparse point clouds. Ultimately, they both achieve the common goal of providing smoother feature maps and attention maps for downstream visual processing. We agree that integrating these ideas will further improve our framework. We will consider studying registration markers as a promising extension in future work.

---

Question 2: Additional comparison methods.

We thank the reviewer for pointing out related occlusion-oriented pretraining works. MSC and Sonata focus on improving pretraining efficiency and mitigating the geometric shortcut problem, respectively. Both methods combine reconstruction and contrastive objectives, demonstrating excellent performance in 3D scene understanding. BlindFormer specifically addresses attentional blindspots by dynamically reallocating attention to under-explored regions, improving robustness to occlusion and sparsity. We view our blindspot mining as complementary rather than competitive and will include a discussion of MSC and Sonata to the related work section in the final revision.

---

Paper Formatting Concerns: Formula optimization.

Thank you for your suggestions for revisions to the formulas and symbols. We have re-reviewed the entire paper and corrected some minor writing errors to ensure clarity and accuracy in the final version.

We thank the reviewer for the time and the valuable feedback. We believe that we have fully addressed your concerns and will incorporate the points mentioned below into the final version. We would be happy to address any further questions you might have.

---

Weakness 1 & Question 1: Empirical evidence in support of motivation.

We appreciate the reviewer's concern and agree that empirical evidence is crucial. Our evidence is threefold:

(1) Qualitative analysis of attention distribution. Baseline models exhibit highly concentrated attention, with a few points consistently dominating (lines 609-615). BlindFormer decentralizes attention by revisiting blindspot regions. Rather than discarding the original attention, our blindspot mining and joint optimization refine it to obtain a broader and more balanced distribution.

(2) Quantitative analysis of performance under perturbations. Standard Transformers show clear robustness issues in a variety of noisy environments. For example, on OBJ-BG with Gaussian noise $\sigma$=0.03, Point-MAE achieves only 47.2% accuracy, while BlindFormer improves it by 13.4%. This shows that the standard transformer's inherent attention bias exacerbates performance degradation when some regions are noisy or partially missing. Thanks to blindspot mining, BlindFormer encourages the model to exploit a wider range of information cues to solve this problem.

(3) Theoretical justification. Attention adapts gradually and excessive focus leads to sparse features, reducing redundancy and making models brittle. BlindFormer acts as a regularizer, encouraging richer and more balanced perceptual representations, thereby improving generalization and robustness.

---

Weakness 2: Performance improvement.

(1) Consistent improvements across settings. Although individual improvements may seem small, BlindFormer shows consistent improvements over supervised learning and self-supervised backbone models across all benchmarks and tasks, demonstrating its generalizability rather than being tuned for a specific dataset.

(2) Robustness under perturbations. The improvements are more significant under challenging conditions. For example, under Gaussian noise ($\sigma$=0.03), BlindFormer improves the classification performance of Point-MAE on OBJ-ONLY by 17.2%; under rotation perturbations, it improves the segmentation performance of PointGPT-S on ShapeNetPart by 3.3%, demonstrating its value in real-world noisy environments.

---

Weakness 3: Minor writing errors.

Thank you for your careful reading and kind feedback. We have re-reviewed the entire paper and corrected some minor writing errors to ensure clarity and accuracy in the final version.

---

Question 2: The role of perturbation probability.

We would like to clarify the role of dynamic blindspot generation. If masking relies solely on attention scores, it repeatedly targets the same high-attention regions, reducing diversity and causing over-regularization. The perturbation probability adds randomness, ensuring that less-salient regions also have a chance to be masked. This promotes exploration of a wider range of cues. The probability temperature in Table 6(c) in the paper is essentially a balance between the effects of attention probability and perturbation probability. To make it more explicit, the ablation results after removing the perturbation probability are shown in Table A. The consistent drop in performance under this configuration confirms the importance of the perturbation variable in improving generalization and robustness.

Table A. Ablation studies of perturbation probability.

---

Question 3: Symbol explanation.

We apologize for the typo in Eq. 6 — $H^m_i$ should be $H^b_i$, representing the blindspot branch feature. The contrastive loss is a symmetric InfoNCE-style objective between the blindspot ($H^b_i$) and standard representations ($H^s_i$). Its purpose is to align representations learned from complementary attention distributions, encouraging the model to explore blindspot regions while maintaining consistency with the standard view. Thank you for pointing this out, and we will double-check the writing of the paper in the final version.

---

Question 4: Contrastive loss weight.

We would like to explain in detail your doubts about contrastive loss weight. $\lambda$ in Eq. 7 is modulated during training following a two-phase schedule. In the early stage, $\lambda$ is set to 0 so that the model focuses on the original loss $L_{origin}$, ensuring stable representation learning. As training progresses, $\lambda$ is set to 0.6 to incorporate the blindspot-aware contrastive loss $L_{contra}$, enabling the model to align blindspot and standard features. The specific values of $\lambda$ are detailed in the ablation studies Table 6(d), where we show that optimal weighting leads to the trade-off between task performance and blindspot feature alignment.

We thank the reviewer for the positive evaluation and the valuable feedback. We believe that we have fully addressed your concerns and questions, and will incorporate all points mentioned below in the final version. We would be happy to address any further questions you might have.

---

Weakness 1: Performance improvement.

We acknowledge that the +17.2% gain on OBJ-ONLY stems from a challenging baseline scenario, but this does not diminish its significance. The low baseline highlights the brittleness of standard Transformers under noise, where BlindFormer offers a substantial robustness improvement. Especially, we achieve these gains without architectural modifications or additional inference parameters. BlindFormer demonstrates consistent gains across all datasets and tasks, even when baseline performance is already strong, showing that the approach is not limited to extreme cases. This consistency across diverse conditions underpins the method’s generalizability and practical relevance.

---

Question 1: The original loss in ablation studies.

In our ablation studies, $L_{origin}$ corresponds to loss in the original pre-training method (e.g., the reconstruction loss for Point-MAE). To evaluate the effect of blindspot mining strategies, we incorporate both Random Blindspot Mining (RBM) and Attentional Blindspot Mining (ABM) into BlindFormer’s dual-branch framework. RBM randomly masks a fixed proportion of points to simulate blindspot discovery without attention guidance. In contrast, ABM leverages the attention distribution to suppress salient regions to encourage learning from blindspot cues. They both ensure that the model needs to rely on the features of the mined blindspot region to reconstruct the complete point cloud, and calculate the original loss through $l_2$ Chamfer distance.

---

Question 2: Extension of our approach.

We believe that the strategy of blindspot mining can be generalized to other Transformer-based models, but its impact depends on domain characteristics. Due to the lack of redundant information, over-focusing on a few salient regions is particularly harmful in point clouds. This motivates the emergence of BlindFormer, which reallocates attention to explore blindspot and capture more diverse geometric clues.

In domains such as images or natural language, when redundant information (irrelevant background) significantly deviates from the subject, high attention to core semantics is often beneficial. When they are somewhat related, occasionally suppressing these regions during training may encourage the model to exploit complementary context. Overall, our method still plays the role of attention regularization. By dynamically mining blindspot regions, the model can better exploit complementary information, thereby improving robustness to occlusion and noisy inputs.

---

Question 3: Phenomenon understanding and strategy improvement.

(1) BlindFormer complements rather than opposes Standard Transformers. Standard self-attention often over-concentrates on a few dominant cues, which can lead to overfitting and reduced robustness, especially for point clouds with limited redundancy. By dynamically masking salient regions, BlindFormer redistributes attention, encouraging integration of complementary under-attended cues for a more balanced representation. Especially, it does not completely negate the original attention bias - a more robust attention distribution is obtained while preserving the original task and representation contrast.

(2) Other strategies to improve the Transformer. Prior work explores techniques such as attention dropout[1], label pruning[2], and loss-based regularization[3]. BlindFormer offers a principled, data-driven alternative by explicitly leveraging the model’s own attention distribution to identify and learn from blindspots, yielding more generalizable and robust point cloud representations.

[1] Wu, Qiangqiang, et al. "Dropmae: Masked autoencoders with spatial-attention dropout for tracking tasks." CVPR, 2023.

[2] Park, Dongmin, et al. "Robust data pruning under label noise via maximizing re-labeling accuracy." NeurIPS, 2023.

[3] Zhai, Shuangfei, et al. "Stabilizing transformer training by preventing attention entropy collapse." ICML, 2023.

---

Limitation 1: Description of limitations.

We appreciate your suggestions on limitations. As mentioned in lines 624-631, exploration in more tasks and architectures will further demonstrate the generalizability of our method.

Furthermore, we carefully checked all experiments and found that BlindFormer may not improve every category equally. For example, in OBJ-BG classification under Gaussian noise, although overall accuracy improved, a few categories with already highly effective original attention experienced slight drops (e.g., “chair” from 91.03% to 87.18%, “table” from 87.04% to 85.19%), likely due to attention redistribution reducing reliance on original salient cues. However, BlindFormer enables stronger generalization and robustness, leading to significant gains in categories where the original attention struggled (e.g., “bin” from 47.50% to 80.00%, “door” from 7.14% to 57.14%) and overall improved performance.

We thank the reviewer for the positive evaluation and the valuable feedback. We believe that we have fully addressed your concerns and questions, and will incorporate all points mentioned below in the final version. We would be happy to address any further questions you might have.

---

Weakness 1 & Question 1: Evaluation on other tasks.

We have evaluated object detection on ScanNetV2 by replacing 3DETR's encoder with our pre-trained encoder. Although there is a performance gap with methods dedicated to scene understanding, BlindFormer enhances scene comprehension of baseline through dynamic blindspot mining.

Table A. 3D object detection on ScanNet v2.

---

Weakness 2: Symbol explanation and effect.

We appreciate the reviewer’s suggestion and agree that the notation could be clearer. $H^m_i$ in Eq. 6 is indeed a typo and should be $H^b_i$, representing the blindspot branch feature.

The two terms in Eq. 6 enforce bidirectional alignment: blindspot to standard and standard to blindspot. Regarding the two terms, we experimented with removing each of them, and the results are reported in the Table B. This bidirectional alignment design prevents representation collapse, ensuring that the standard branch learns from under-attended cues while the blindspot branch remains grounded in the original representation, resulting in richer and more robust features.

Table B. Ablation studies of contrastive loss.

---

Weakness 3: Positive and negative pairs.

We thank the reviewer for the suggestion. Our contrastive loss in Eq. 6 uses both positive and negative pairs, where positive pairs enforce alignment between blindspot and standard branch features, and negative pairs ensure discrimination from other samples. Following your kind suggestion, we conducted an ablation in Table C comparing the use of positive pairs only versus both positive and negative pairs. The results confirms that both positive and negative pairs are essential for robust blindspot-aware representation learning.

Table C. Ablation studies of positive and negative pairs.