From Sparse to Dense: Learning to Construct 3D Human Meshes from WiFi

There are several weaknesses in the paper in its current form:

1. For the greater part of the ICLR audience WIFI signals are not a daily business. The paper lacks an intuitive explanation of what information is contained in CSI and what’s not. The paper talks a lot about tensor dimensions which are not clearly justified and explained. The inputs and outputs of each component of the models do not become clear.
2. There are not enough details about the hardware topology and the used bandwidth. There is only a statement “…a bandwidth of 20/40MHz…” in Section B. However, you did not describe, which bandwidth is used. Also, a detailed discussion, why a radio system with only 20/40MHz can achieve such high accuracies is missing. Such high localization results can even not be achieved by active, WIFI based localization systems, also working with OFDM signals. Why should your passive approach be even better than those?
3. We also miss a comparison of other algorithms on your dataset, as well as your method evaluated on publicly available datasets. The results shown in Table 1 cannot be used to compare your results to the state of the art, this is unfair. The accuracy is highly correlated to the quality of the dataset. We acknowledge difficulty in comparing the datasets, but WiTR could still be compared with one of the single-person methods on the dataset that the paper proposes.
4. The paper aims at providing novelty in mesh reconstruction. At the algorithmic level however, there is not much contribution.
5. It is unclear how such a system works out of the lab. I am less concerned about real-time processing but the CSI data itself. How well does it generalize to unseen trajectories, movements etc. It is totally unclear how the training and test data are organized.

Some minor comments:

• “…which typically consists of only about 10,000 elements if…” - This depends highly on the used radio system, employed bandwidth and topology. There is no “typical” size of a CSI.
• “...meshes (with a size of N × 6890 × 3) representing…” – What is “N”?
• “…which is at the same level as previous image-based and radar-based methods” - I cannot find any comparison to such methods in your work. The accuracy highly depends on the environment and scenario. A direct comparison of accuracies is not useful.
• “…random phase drift and flip.” - You may explain what a phase flip is.
• “…linear transformation to denoise the phase signal following…” - How did you deal with the phase shift and drift among the four network cards?
• “…CSI signals Z ∈ C3×3×20×30 are complex signals.” - What do the individual dimensions stand for?
• “We propose a new way of understanding the CSI signals, where…” - Your understanding of the CSI is not in line with the communication theoretical definition.
• “…collects CSI tensors ∈ C3×3×30×300 per second.” - What is the last dimension and why do you only use 20 samples of the 300 and which one of them? You must explain the recorded data detailed and reason why you can just crop the dimension.
• How big is the label noise resulting from the employed camera-based mesh regression?
• What is G in Fig. 2? It seems to be the number of coarse poses but how is it defined?
• While Geng et al. 2022 in general being different from the proposed work (i.e., not working on mesh models) it would be beneficial to also differentiate from it in the paper.
• Intro: “We present a[-n-] fully end-to-end
• Sec. 2: “body mesh.[ ]One”
• Sec. 2: what is a top-down method?
• Sec. 3.: “sensing with WiFi ”
• Sec. 3: “coarse decoder[ ](i.e.,…”
• Sec. 3: “identity token.[ ]We”
• Sec 3.: [w]here after Equation 2
• Table 1: it is not immediately clear what the unit is for the mentioned results

Appendix A

How did you separate Training and Test data? Are they different recordings on different days or just random splits? A random split is not a valid evaluation. You have to share more details about the generation of the datasets to emphasize the generalization abilities and high localization results.

1. The technical novelty is not sufficient. The proposed method for WiFi-based mesh construction relying on camera-based supervision has been well explored [1][2][3].
2. The proposed method seems to be overfitting to one environment. It is not realistic to deploy multiple pairs of WiFi for this specific task.
3. The experiments are conducted in a self-collected manner, which is good, but for AI conferences, public datasets including more environments should be utilized to validate the effectiveness of this work, e.g., the MM-Fi [4].

[1] Geng, Jiaqi, Dong Huang, and Fernando De la Torre. "DensePose From WiFi." arXiv preprint arXiv:2301.00250 (2022). [2] Wang, Yichao, Yili Ren, Yingying Chen, and Jie Yang. "Wi-Mesh: A WiFi Vision-based Approach for 3D Human Mesh Construction." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 362-376. 2022. [3] Wang, Yichao, Yili Ren, Yingying Chen, and Jie Yang. "A wifi vision-based 3D human mesh reconstruction." In Proceedings of the 28th Annual International Conference on Mobile Computing and Networking, pp. 814-816. 2022. [4] Yang, Jianfei, He Huang, Yunjiao Zhou, Xinyan Chen, Yuecong Xu, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, and Lihua Xie. "MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing." NeurIPS-23 Datasets and Benchmarks Track

This paper uses a camera-based method to generate labels from RGB videos. Although the paper states that nearly 20 students were recruited to manually screen the dataset frame by frame for almost a week, the quality of the dataset can still be questioned. No other methods were used for experiments on the proposed dataset except for WiMTR to evaluate the quality of the proposed dataset.

The motivation of the work is unclear. This paper does not fully emphasize the differences between this work and the single-person mesh regression method and dataset from WiFi, such as Wi-Mesh[1]. The scenes of the proposed dataset are simple. The three scenes are very similar, with a small number of people scattered. Besides, there is no interaction between people and the distances between people and sensors are also very close. The design and results of experiments do not fully reflect the strengths of multi-person.

There are some imprecise parts in this paper. For example, the paper mentions that the shape parameters are generated in the Coarse Decoder stage but does not mention how to generate them. This paper uses 𝜃 to represent shape parameters and 𝛽 to represent pose parameters, which are contrary to the conventional representations.

The authors organized related work based on the type of sensors. However, LiDAR-based MoCap is missing. For example, LiDARCap[2] is a LiDAR-based human motion estimation method.

Grammar mistakes exist. For example,‘ We present an fully end-to-end WiFi-based Mesh Regression Transformer’ in the second point of main contributions in the introducition part should be ’ We present a fully end-to-end WiFi-based Mesh Regression Transformer’.

References [1] Yichao Wang, Yili Ren, Yingying Chen, and Jie Yang. Wi-mesh: A wifi vision-based approach for 3d human mesh construction. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 362–376, 2022. [2] Li et al., LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds, CVPR 2022

1. The novelty of the proposed multi-person mesh regression transformer is somewhat limited as it follows the same workflow as PETR [1], a prior work for multi-person pose estimation.

2. This paper is not written clearly in general and it causes some confusion while reading (see questions).

[1] Shi, Dahu, et al. "End-to-end multi-person pose estimation with transformers." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.