We sincerely appreciate your positive feedback regarding the novelty and effectiveness of our method and your recognition of the detailed and well-structured writing. In the following, we address your questions point by point.

• In Eq. 1, what is Linear? Is it a neural network? If so, is it optimized in the training of the whole pipeline?

Linear means a linear projection layer responsible for estimating 2D joint to 3D anchor offsets $\\delta_J$from the input 2D pose $P\_J^{2D}$, and is optimized end-to-end during the training process.

• For the plane of root joint in Line 151, is it fixed to z=0?

Yes, follow the common practice of 3D human pose estimation methods (CA-PF $NeruIPS'23$ , HiPART $CVPR'25$ , KTPFormer $CVPR'24$ etc.), we set the root joint depth $z$ to 0. We will add the explanation regarding this in Line 151.

• In the training, are the GT 2D points or the estimated 2D points used?

We strictly follow the same input as the previous methods (CA-PF $NeruIPS'23$ , HiPART $CVPR'25$ , KTPFormer $CVPR'24$ etc.) for fair comparison, i.e., using the same estimated 2D pose for training and testing (Line 269-272).

• In Eq. 6, what are these sampled points? How are they generated? And how to estimate the offset?

$DCA(a) = \\sum\_{n \\in N} W\_n \\phi(F\_{3D}, P\_a + \\Delta S\_n)$ For an input anchor query $Q\_{anchor}$ and corresponding anchor position $P\_a$, we treat $P\_a$ as reference point to execute deformable attention $1$ . Aligned with $1$ , we equip each anchor with $N$ sampling points nearby to dynamically interact with $F\_{3D}$. These sampled points are generated by adding sampling offsets $\\Delta S\_n$to the reference points $P\_a$ (i.e. 3D anchor positions). The sampling offsets $\\Delta S\_n$ are predicted from the input queries (also referred as anchor queries $Q\_{anchor}$ in our paper) through a linear projection layer. This design enables the Transformer to dynamically focus on sparse but semantically meaningful local regions in an adaptive manner, improving both efficiency and accuracy. Due to space limitations, we will add detailed explanations for Eq.6 in the revised version, and add a flowchart for the signal to enhance clarity.

$1$ Deformable DETR: Deformable Transformers for End-to-End Object Detection

• What is the noise level in Table 9? The unit of noise scale is not clear.

The noise level in Table 9 is measured in pixel units, based on images of resolution 256×192.

• Providing some qualitative results on 3DPW would benefit the evaluation, as it has GT and is in less constrained outdoor setting.

We agree that qualitative results on 3DPW would be valuable for demonstrating the effectiveness and generalization of our method, especially in unconstrained outdoor settings. Unfortunately, due to the constraints of the rebuttal policy, we are unable to include additional visualizations at this stage. We will include more qualitative results on 3DPW in the revised version to further support the evaluation. In the meantime, to demonstrate the generalization ability of outdoor scenes, we present the comparison results of MPI-INF-3DHP with SOTA and GT (Figure 11) as well as the qualitative results of in-the-wild internet images (Figure 12).

• For other methods shown in Table 3, are they trained on 3DPW?

The methods listed in Table 3 are all 3D human pose estimation approaches that follow the cross-dataset setting (training on Human3.6M and testing on 3DPW). The performance metrics of these methods are taken from their latest publicly available reports. We compare against them under the same evaluation protocol to ensure a fair comparison. Compared to the latest SOTA method HiPART (CVPR 2025), we achieve significant improvements in performance (74.9 vs 77.2, 2.3 $\\downarrow$), demonstrating the strong generalization capability of our approach.

• Since the experiments are conducted with the 2D pose estimator $33$ , I'm wondering whether the results will change a lot if using different pose estimators？

We select three widely used 2D pose estimators (i.e., CPN, HRNet-w32, HRNet-w48) for ablation experiments to examine how predicted 3D pose quality varies with input 2D accuracy. The results are listed below

The accuracy of PandaPose remains relatively stable with respect to the errors of the 2D pose estimator. Compared with CPN and HR-Net-w48, the 2D accuracy (mAP) decrease by 5.5%, but the error of PandaPose only increase by 1.5%. Meanwhile, in Figure 9 of the main text, we gradually add Gaussian noise to the 2D pose to study the method tolerance to noise levels and our method maintains the best anti-noise ability. When the pixel-level noise scale ranges from 0 to 5, the error of PandaPose only increases by 16% (from 39.8 to 48.2), while the image-based CA-PF increases by 56% (from 41.4 to 64.9), and the sequence-based SOTA KTPFormer increases by 112% (from 40.1 to 86.9). The noise resistance can be attribute to different 3d anchor setting and anchor-to-joint ensemble prediction mechanism.

We appreciate the recognition by Reviewer HbVK of our writing, well-motivated design, and empirical results. We will address all concerns regarding weaknesses and questions in the following.

Depth Discretization Strategy

Thanks for the insightful suggestion. We simply replace the classification depth head with a regression head for comparison as shown in table below. The classification approach achieves better accuracy, with the optimal bin number being 64. The improved performance can be attributed to the better alignment between the task complexity and the lightweight classification head, which helps alleviate fitting difficulty — a benefit particularly evident in challenging cases. We also observe that using too many bins can degrade performance, which aligns with findings in previous lightweight depth estimation methods $1,2,3$ . We will include these results and discussions in the revised version to further clarify the advantages of our method. We kindly note that our insight regarding the depth head does not lie in whether it is for classification or regression, but rather in the joint-wise approach. Ordinary predicting single depth map in principle cannot reflect joint depth in cases of self-occlusion (as shown in Figure 5 of the main text). The proposed joint-wise module addresses this issue by independently predicting the depth distribution for each joint, improving robustness as shown in Table 5 row2 - row3 of main text. Especially under challenging cases with significant occlusion, the predicted MPJPE decreased by 2.8mm (75.9 - 73.1).

$1$ Deep ordinal regression network for monocular depth estimation, CVPR2018

$2$ Adabins: Depth estimation using adaptive bins, CVPR2021

$3$ idisc: Internal discretization for monocular depth estimation， CVPR2023

Computational cost

In Table 7 of the appendix, we compare the parameters and FPS with the SOTA image-based method CA-PF, as shown in the following table (assuming real-time application requires the single-frame input). Our method demonstrates significantly superior accuracy, especially in challenging cases, while FPS and inference consumption remain at a similar level to CA-PF. Following their claim of efficiency, PandaPose also meets the real-time requirement.

Number setting and semantic characteristics of 3D anchor

Regarding the the number setting of 3D anchors, we determine it based on the trade-off between performance and computational overhead. We conduct ablation experiments under different anchor numbers (see the table below) and find that increasing the anchor number from 149 to 596 continuously improve performance; however, when it is further increased beyond 596, the performance improvement become limited, while the computational overhead increase significantly. Therefore, we finally set the anchor number to 596 as the balance point between performance and efficiency.

Regarding the semantic characteristics of anchor, although we are unable to present the visualization results of anchor in the paper due to the rebuttal policy, we have verified through quantitative and qualitative analysis whether its spatial distribution has semantic significance. Specifically, we qualitatively and quantitatively observe the distribution and contribution of different types of anchors (adaptive local anchor / fixed global anchor) to the final joint prediction (as shown in Figure 3 of the main text). By selecting top-50 anchor-to-joint weights, we identify informative anchors contributing most to joint prediction. (Line 139-141) The distribution of the final informative anchors shows obvious clustering for the target joint (mean offset 73.4mm), with offsets to other joints all significantly larger than this value (mean offset 161.9 mm), demonstrating the specificity of the distribution. We also analyze the changes in the offset of each anchor to the corresponding joint during the training process. This offset shows a clear downward trend during training as shown in the table below, indicating that the model gradually learns the accurate correspondence between anchors and 3D joints. This further demonstrates that anchors are not randomly distributed but gradually converge to positions consistent with the semantics of the human body structure.

In summary, our analysis shows that the learned 3D anchors are not only effective in terms of performance but also possess certain semantic significance. We will further supplement these analyses in the revised manuscript to more clearly demonstrate the design motivation and characteristics of the anchors.

2D feature sampling stratery

Thanks for the insightful comment. We have considered this issue and implemented targeted optimizations in the design of feature sampling strategy and anchor mechanism:

1. For feature sampling strategy, based on the four-level multi-scale feature maps after downsampling, we perform feature sampling only on the lower two levels while preserving global semantic feature from two lower resolution levels (Line 205-207). Since 2D pose coordinates are inherently less sensitive to prediction errors after downsampling, we also enhance robustness to localization inaccuracies by sampling features within an r×r neighborhood around each 2D joint (Line 202-203).

     To further demonstrate the optimizations above, we conducte an investigation into the similarity of visual features and the impact on final performance under noisy 2D pose inputs as table below. The resolution of the image is 256*192. When the noise scale is controlled within a certain range (0-5 pixels), the sampled image feature similarity can maintain relatively high stability (100% - 95.21%). When noise overflows to a large extent (10 pixels), the image features will deviate to a certain degree (85.33%) but still at a relatively high similarity.

2. Apart from the optimization of the feature sampling strategy, we additionally adopt the anchor mechanism based on local and global anchors to ensure that our model can provide reliable 3D predictions even when there are significant errors in 2D pose estimation.

   • For Local Anchors, we sample a set of 3D local anchors for each 2D joint using learnable sampling offsets. As shown in Figure 10 of the main text, even when 2D pose estimations exhibit substantial deviations due to motion blur or severe self-occlusion, these local anchors still effectively capture the positions of 3D joints.

   • For Global Anchors, we established a series of fixed global anchors that cover the potential human pose space (Line 149-152). This ensures that, despite significant shifts in local feature details caused by pose estimation errors, the global anchors can serve as stable reference points to provide global context, thereby maintaining the quality of the final 3D pose estimation.

   • Additionally, our method does not solely rely on a single offset prediction, but instead adopts an ensemble approach for offset prediction. This means that for each anchor, we generate multiple offset predictions and determine the final position through weighted averaging (Eq.7). This method not only improves the robustness of the prediction but also effectively reduces the overall performance degradation caused by inaccurate individual predictions.

To demonstrate the effectiveness and superiority of the above optimizations. We compare with other SOTA methods to test the robustness to noisy inputs (see table below and Figure 9 in main text). When the noise scale is 10 pixels, our MPJPE will decrease by 63.8%, but the magnitude is still not as severe as that of CA-PF by 151.9% and MixSTE by 284.8%.

Finally, we would like to thank you for the typo in Fig13, and we will update this to Table 6 in the revised version.

We would like to thank Reviewer G3sQ for the recognition of innovations, writing clarity and excellent performance. We also sincerely appreciate your attention to the anchor-based design in our method. We will comprehensively address your concerns from multiple perspectives: paradigm comparison, principle analysis spanning distribution, training stability, and ensemble mechanism, as well as relevant quantitative results.

1. Inclusiveness of the paradigm: directly predicting the offset of the 3D root node can be seen as a special case of our method. Anchor-based PandaPose initially sets global fixed anchor and the joint-wise local anchor, then combines the two types of anchors to regress the offset for obtaining 3D joint predictions. When the joint-wise anchor is discarded and the global fixed anchor is set to the root node with its number decrease to 1, PandaPose degenerates into the method you describe. We quantitatively evaluated this strategy, yielding an MPJPE of 42.7. Our final experimental results showed a 6.8% reduction in error compared to this baseline.

In the revised version, we will supplement this comparison to Table 4 in the main text to further improve the completeness of the experiment.

2. Analysis of the essential principle:

   • The global fixed anchor and joint-wise anchor we propose efficiently deliver a robust initialization for 3D pose lifting that is closer to the target joint distribution compared to the root node. Subsequently, the final prediction can be obtained by simply predicting an offset with a smaller numerical range ( 91.3mm v.s. 456.2mm in world coordinate system, based on average offset statistics). This substantially lowers the difficulty of directly predicting from the 3D root node to the target 3D joint.

   • On the other hand, a smaller predicted offset also makes network training more stable and the convergence speed faster. Specifically, under the exact same dataset (Human3.6M) and computing environment, aligning the training settings, when the training MPJPE loss converges to 60mm, our method requires about 280 iterations, the root node strategy requires about 510 iterations, and CA-PF requires about 1350 iterations.

   • Additionally, our anchor mechanism essentially serves as an ensemble prediction for the joint, generating the coordinates of the target joint by integrating the offset/weight of multiple anchors. This characteristic of "ensemble" makes the overall prediction more fault-tolerant to local noise, which has also been widely proven effective for performance improvement [1] and robustness against outliers [2].

We will also supplement the paper with the detailed discussion above to more comprehensively explain the advantages of our motivation and anchor design.

3. Justification of $Q\_{anchor}$. The initial $Q\_{anchor}$ is derived from the 3D anchor set. We map it to a high-dimensional space through a linear layer, thus encoding it as a learnable anchor query, which is used to interact with RGB & depth features in the decoder. This is also mentioned in Line 221-224 of the main text.

$1$ ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers

$2$ Uncertainty-based offline reinforcement learning with diversified q-ensemble

We appreciate Reviewer MoUK recognizing the novelty and superior performance in our method. Below, we address your concerns from two perspectives:

Efficiency analysis

We supplement the GFLOPS metric and inference memory cost based on Table 7 in the appendix. Our method achieves superior performance with significantly fewer FLOPs compared to sequence-based MixSTE. Compared to the previous image-based SOTA CA-PF, although our method has slightly higher GFLOPs (still at an absolutely low level compared with MixSTE), it benefits from the highly parallel design of the attention implementation, resulting in comparable FPS and GPU memory cost. Moreover, our method demonstrates advantages in overall accuracy, especially in challenging cases, indicating its high efficiency and effectiveness. Due to the limitations of the rebuttal policy, we can only provide tables instead of scatter plots, and we will add scatter plots corresponding to the above data in the revised version. Overall, PandaPose achieves the best accuracy-efficiency balance.

Tolerance to 2D pose estimation inaccuracies

We also thank the reviewer for the advice on evaluation of the tolerance to 2D pose estimation inaccuracies. We select three widely used 2D pose estimators (i.e., CPN, HRNet-w32, HRNet-w48) for ablation experiments to examine how predicted 3D pose quality varies with input 2D accuracy. The results are listed below

• The accuracy of PandaPose remains relatively stable with respect to the errors of the 2D pose estimator. Compared with CPN and HR-Net-w48, the 2D accuracy (mAP) decrease by 7.7, but the error of PandaPose only increase by 1.5%, while CA-PF increase by 4.5%.

• Meanwhile, as mentioned in Lines 262-263, our criterion for selecting challenging cases is based on 2D pose prediction error, i.e., these cases are the worst-performing part of 2D pose prediction in the dataset. We outperform CA-PF by a significant margin on the MPJPE in this subset (73.1 v.s. 82.4, 12.1↓). This also demonstrates our better tolerance to 2D noise.

• Moreover, in Figure 9 of the main text, we gradually add Gaussian noise to the 2D pose to study the method tolerance to noise levels and our method maintains the best anti-noise ability. When the pixel-level noise scale ranges from 0 to 5, the error of PandaPose only increases by 16% (from 39.8 to 48.2), while the image-based CA-PF increases by 56% (from 41.4 to 64.9), and the sequence-based SOTA KTPFormer increases by 112% (from 40.1 to 86.9). The noise resistance can be attributed to the design of anchor ensemble mechanism based on local and global anchors with ensemble joint prediction.

In conclusion, PandaPose achieves state-of-the-art robustness to input 2D poses, which stems from our effective 3D anchor setup and anchor-to-joint mechanism.

We appreciate your careful reading of our manuscript and your comments on the writing. Actually, compared with previous methods, PandaPose has many differences in both the overall framework and specific design implementation. Due to limited space, we spend considerable effort in the introduction to help readers quickly establish the logical flow (problem - motivation - method - results), and provide further details in later sections. In the future, we will strengthen the connection between the concepts in the introduction and Figure 1, optimize the layout, and add explanatory content to eliminate misunderstandings (if accepted, there will be an additional page). Meanwhile, our writing has been generally well-received by the other four reviewers, with an average score of clarity above 3. Next, we will address all your concerns below.

Introduction writing

For Line 43-45, we aim to clarify that the overall architecture is built on a more advantageous initial distribution of 3D anchors and the anchor-to-joint mapping approach. This is the key innovation and main advantage of our method. Line 48-51 then detail the design of the 3D anchors. Line 57-59 and 61-65 summarize the introduction, analyze the design concepts and motivations, and explain the method's superiority in handling challenging samples, as well as its efficiency and accuracy. We expect readers to grasp important concepts such as 3D anchor space, anchor-to-joint mapping, and joint-wise depth early in the introduction.

LayerOut and Figure

Following the comparison protocol as previous works (CA-PF, MixSTE etc.), Figure 11, as a qualitative comparison, highlights the parts our method has better prediction in circles as in the caption. Figure 12 serves as an OOD experiment to tests the generlization. Figure 10 is dedicated to demonstrating the robustness of the proposed joint-wise anchor in the case of 2D pose prediction errors as in Line 323 and the caption. All figures in paper are vector graphics and carefully designed. It is recomended to view the paper through a standard PDF reader and zoom in on the illustrations for clearer observation of details. We will enlarge some images into full-width figures in subsequent versions for better viewing.

Complexity of decoder

Regarding the decoder design, our approach is an improvement upon the standard deformable Transformer decoder $1$ (comprising self-attention and deformable cross-attention). Specifically, we introduce a depth-domain cross-attention module and extend the original 2D deformable cross-attention to 3D deformable cross-attention. It is worth emphasizing that the deformable attention mechanism has been widely validated for its effectiveness across numerous computer vision tasks. Its successful application is also evidenced by pose estimation related works such as DeforHMR $2$ (3DV 2025) and DeFormer $3$ (CVPR 2023), providing a solid foundation for our design choice. We further supplement the ablation experiment on the improvement of prediction by the improved components as table below. Both modules are designed to model the depth domain, and experiment proves that they can significantly enhance 3D pose prediction quality, as reflected by the decreasing MPJPE.

[1] Deformable DETR: Deformable Transformers for End-to-End Object Detection

[2] DeforHMR: Vision Transformer with Deformable Cross-Attention for 3D Human Mesh Recovery

[3] Deformable Mesh Transformer for 3D Human Mesh Recovery

Joint-wise Depth Clarification

Overall, our approach emphasizes the necessity of accurate joint-wise absolute depth for 3D pose estimation, differing fundamentally from the methods you mentioned. SMAP[1] models root joint depths in multi-person scenarios without estimating specific joint absolute depth values, leading to potential error accumulation. The limb depth map method[2] interpolates regressed joint depths based on limb rigidity, limiting accuracy due to interpolation errors and failing in non-rigid motions. HEMlets[3] encodes only relative depth ordering without quantitative depth values, which is insufficient for precise 3D coordinate calculation. In contrast, our method directly estimates and utilizes joint-specific metric depths, avoiding error accumulation and capturing local depth variations caused by joint movements. Unlike SMAP's reliance on a long estimation chain prone to noise and limb depth maps' dependence on rigid limb assumptions, our approach ensures robust and high-precision 3D positioning through direct quantitative depth estimation.

Moreover, the aforementioned methods lack theoretical analysis on joint-wise depth estimation. In Section 3.3, we provide a detailed theoretical and technical justification of our approach—setting us apart from previous works. Through qualitative and quantitative analysis (Figure 5, Figure 6, Table 5), we clearly articulate the motivation behind our design. Technically, our method introduces a lightweight depth encoder that predicts a joint-wise depth map with real physical values in a classification manner—distinct from existing approaches. This provides a more accurate and robust depth prior, enabling more effective feature interaction in the subsequent decoder.

[1] SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

[2] 3D Human Pose Estimation via Explicit Compositional Depth Maps

[3] HEMlets Pose: Learning Part‑Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Result on 3DPW

Our experimental setup is specifically designed to evaluate cross-dataset generalization—training on Human3.6M and testing on 3DPW. This protocol is widely adopted to assess model robustness under domain shifts, such as differences in camera setups, subjects, and environments. In Table 3, all compared methods are SOTA approaches evaluated under the same cross-dataset setting, focusing on single-person 3D human pose estimation. We follow mainstream practices in our comparisons, and the performance metrics for all cited methods are taken directly from their publicly available results—including latest HiPART (CVPR 2025), ensuring both fairness and up-to-date benchmarking. We emphasize that our goal is not to establish new absolute performance records on 3DPW spanning different settings and methodological paradigms, but rather to evaluate how well models generalize from controlled, lab-based settings (Human3.6M) to more challenging, in-the-wild scenarios (3DPW). This evaluation paradigm is highly relevant in real-world applications, especially when labeled in-the-wild data is limited or expensive to acquire.

It is worth noting that many mesh-based methods achieving strong performance on 3DPW are trained on large, mixed datasets that include in-the-wild or synthetic data. A direct comparison under such conditions would be unfair. To enable a more equitable comparison with mesh-based approaches, we retrain TokenHMR (CVPR 2024) under our same experimental setup. Under this setting, TokenHMR achieves an error metric MPJPE of 78.5 on 3DPW, which is higher than our result of 74.9, further demonstrating the effectiveness of our method in this generalization scenario.

Clarification of Human3.6M

In response to the reviewer's doubts about whether the results on Human3.6M are practically significant, we would like to clarify that Human3.6M remains a widely recognized standardized benchmark in the field of 3D pose estimation, with advantages in high-quality and controllable annotations. Recently published papers still use this dataset as a core benchmark ( HiPART[CVPR25], UPose3D[ECCV24] etc.). We also observe that the metrics of current 3D human mesh methods on Human3.6M are often inferior to those of 3D human pose estimation (as shown in the table below).

More importantly, although Human3.6M is a relatively "clean" dataset, we observe that existing pose methods still perform poorly in occlusion scenarios within this dataset. For instance, we select complex scenarios in the dataset by measuring 2D pose error > 5(line 262-263 in main text), which often represent self-occlusion and motion blur. Existing methods often lose information about key body parts, leading to a decline in overall pose estimation accuracy. This observation is precisely the motivation and core contribution of our work. Our method demonstrates significant performance improvements in these challenging subsets. For instance, on challenging subset (~5% in the full test set), the MPJPE of ours is 9.3 mm lower than that of the current SOTA method CA-PF (73.1 vs 82.4) as shown in Table 1 of main text.

Minor issues

1. Pose prior：In our paper, we adopt the term "Pose prior" following the convention of previous related works (e.g., C3P[1] [ECCV2022], DiffPose[2] [CVPR2023]), which also used this expression in similar contexts. In these works, "prior" does not strictly refer to prior knowledge or distribution in the Bayesian sense, but more broadly refers to the initial pose hypothesis derived from the 2D joint positions in the input image. This usage aims to emphasize the process by which the model uses the known 2D information as a basis to predict 3D poses. We will also strive to supplement further background in the revised version to eliminate ambiguity.

2. Canonical coordinate system：The canonical coordinate system in the text refers to the 3D world coordinate system with actual physical values, rather than the 2D pose coordinate system. The 3D anchors are located in the same coordinate system with 3D human pose. We will also further strengthen this concept in the paper.