We sincerely thank the reviewer for their constructive feedback. Below, we answer some concerns.

Q1. The motivation is vague - what practical, efficiency and cost advantages do RF signals offer over camera-based 3D reconstruction?

We think there are a number of advantages, and based on your comment, we realize we should add them to the introduction of the paper:

1. Privacy: Using cameras to infer floorplans raise privacy concerns inside homes and offices. Apple’s ARKit (which uses the phone camera or LIDAR to perform localization and mapping at homes) have received pushback from customers. Amazon Alexa has not adopted vision techniques due to privacy. RF based methods learn only the geometry of the environment, without “seeing” the details in rooms. Several companies have expressed enthusiasm to us for RF based imaging solutions.
2. See through: Extensions of EchoNeRF may be able to do X-ray vision, i.e., infer objects inside an opaque box using RF measurements from outside the box. This can have applications in medical imaging, airport security, or military applications (where a team of soldiers may want to infer the floorplan and people inside an enemy building).
3. Fewer measurements: The RF modality can serve as an assistance to cameras to reduce the number of measurements. For instance, cameras may be used to image the lobbies and corridors of a hotel, while the RF modality can sense the rooms and un-visited areas. This lowers the overhead of visiting every location with the camera.

Lastly, we are curious about what can be achieved with RF-based NeRFs and perhaps more applications would emerge once such capabilities have matured.

Q2. The voxel-selection strategy for reflections needs more details. Summing reflected power by iterating over every voxel scale poorly as the voxel count grows. Although the authors accelerate this with discrete orientation angles, further implementation are needed.

Thanks for understanding our paper correctly and asking this question. We make 2 observations:

1. For each discrete orientation, the manifold of reflection is sparse even in 3D. As a result, the number of optimization variables do not grow too fast. For instance, let’s consider all voxels at $90^\circ$ orientation, i.e., their surface normal is perpendicular to the horizontal floor. Now, say a Tx and Rx are on the floor and we want to identify which of the $90^\circ$ voxels above the floor will result in a valid reflection from the Tx to the Rx. For this case, observe that the manifold is the vertical line bisecting Tx and Rx; only voxels on this vertical line are candidates for reflection. Similar manifolds will exist for other discrete orientations.
2. Yes, even if the manifolds are thin/sparse, they may still add up when number of discrete orientations grow. However, observe that we only consider voxels with high opacities ($\approx 1$), since they are the ones likely to reflect. Voxels that are transparent are ignored, implying that we are discarding many voxels even on the manifolds. This also helps the optimization.

Q3. Lack of ablation study. Experiments isolating the reflection model's impact and comparing single-stage versus two-stage training are needed.

Perhaps this confusion arises from not explaining Fig. 5 in detail. Observe that row 4 in Fig. 5 is the output of the LoS model, and row 5 shows the improvements with the reflection model. Row 4 is also the result from the 1st stage of our model (where the LoS model dominates), while row 5 is after completing the full 2-stage training.

If you are curious about training the whole EchoNeRF model in a single stage, we tried that but the one-stage training did not converge and led to poor results. This prompted us to move to the 2-stage solution.

Q4. Proposed method only models line-of-sight plus first-bounce reflections - which only account for less than 6% of total RF power - so it ignores high-order multipath that can be substantial in cluttered or non-planar environment.

We should have been clearer here. It’s actually the opposite. The line of sight (LoS) and the first-bounce together makes up 94% of the total power (and 6% is the contribution from all higher order reflections). The reason for this is that higher order reflections travel longer distances and get absorbed by multiple surfaces; hence their power attenuates significantly by the time they reach the receiver.

Q5. More ablation study should be included, eg. 1) with and without reflection model and 2) only one stage training versus two-stages training.

Please see response to question 3.

We sincerely thank the reviewer for their constructive feedback. Below, we answer some concerns.

Q1. The paper lacks any real-world validation, making it difficult to assess its practicality. This issue is exacerbated by a lack of comprehensive analysis or even discussion of the impact of real-world noise (aside from noise in receiver location in Sec 4.3 and mentioning the use of additive noise in the background on modeling in the appendix).

We acknowledge that EchoNeRF has a considerable simulation-to-real gap. However, even in simulations, this still proved to be a challenging problem (in our opinion), mainly due to the core inverse problem that needed to be solved under unknown number of multipath reflections, sparse signal measurements, and each measurement being a scalar (because RF signal power is a scalar).

We believe this paper has finally solved this inverse problem, arguably the most critical piece in the whole RF-NeRF puzzle. While this does not deliver a practical end-to-end system, the current version of EchoNeRF gives us (and others) a platform to build on with follow-up ideas. These ideas include, but are not limited to, fusing with RGB, audio, LIDAR or other sensors (as suggested by several reviewers); using channel impulse responses (CIR) to extract more information from each measurement; leveraging multiple frequencies in WiFi/6G signals; using floorplan priors; and even post-processing on the final results. These should all help in closing the sim2real gap.

Regarding real-world noise, we have run new experiments this week where noise is added at the receiver. We report graceful degradation in EchoNeRF with increasing noise levels, shown below:

Specifically, we added Gaussian noise to the RSSI measurements with a mean equal to the noise floor (in dB) and a variance of 4 dB. We tested five noise floor levels ranging from -80 dB to -130 dB across the 6 floorplans shown in Fig 5. The SNR at a receiver is computed as the difference between the received signal power and the noise floor (e.g., a received power of -70 dB with a noise floor of -80 dB results in an SNR of 10 dB). Our typical RSSI measurements ranges from -60dB (strong) to -130dB (weak). We report the mean IoU below:

As expected, the performance drops with decreasing SNR; both EchoNeRF_LoS and EchoNeRF’s floorplans are still legible till 30dB, but below that (when noise power becomes comparable to the signal power) both EchoNeRF_LOS and EchoNeRF break down. This is expected.

Q2. The paper claims on L22 that "optical NeRF understandably fails since it is not equipped to handle multipath", which is a misleading generalization, and subsequently misses several works related to optical multipath NeRF that should be cited and/or discussed. Some of the multipath NeRF from lidar literature is similar to this work, but isn't discussed. In particular, PlatoNeRF [1] models first-order reflections from multiple "virtual Txs" and uses a similar two-stage training approach (however, it does not model multiple Txs at once as in this work). A more comprehensive list of papers that should be cited/discussed is below.

In L22, we were referring to vanilla NeRFs which do not model multipath. In L95-L100, we discussed how follow-up work have modeled multipath as a 2-component decomposition problem, while EchoNeRF must handle a $K$-component decomposition, where $K$ is unknown.

Thanks for pointing us to the papers. We will surely cite them in our revised version but the ideas from these papers don’t apply directly to ours. Specifically, the RGB based reflection-aware NeRFs (e.g., NeRFCasting and ORCa) are still focussed on efficiently solving the 2-component decomposition, typically for rendering glossy surfaces or mirrors.

The other line of work using LIDARS (e.g., PlatoNeRF, NLOS imaging) are coping with many unknown reflections, however, LIDARs have very high time resolution (very high clock frequency), and is therefore able to receive the incoming rays in different time buckets. This temporal separation allows the NeRF to make separate measurements for different surfaces in the scene.

Since we use only signal power in EchoNeRF, we only have one single scalar measurement (named RSSI) that contains a mixture of the LoS and all the $K$ reflections. This makes our inverse problem even more complex.

Q3. One of the main advantages of modeling reflections is stated to be occlusion-awareness (L128-129), but this occlusion-awareness is minimally ablated. it's discussed on L263-266, but it's unclear to me why EchoNeRF_LOS cannot see the outer walls as the Txs (red stars) are directly adjacent to them - maybe if the Rx location was shown in the GT, it would become more clear

We understand the confusion and let us briefly clarify it here. We will explain better in the revised version of the paper.

In row 1 of Fig. 5 , the red stars denote the Tx locations, and the light gray dots denote the Rx locations (i.e., where measurements are made). Row 4 of the same figure shows the results from the EchoNeRF_LOS model. The outer walls are not detected in row 4 because there are no Rx location, hence no measurements, from outside the home; the gray dots in row 1 are all inside the home floorplan. However, when there are walls between a red Tx and a gray Rx inside the home, those occlusions show up in row 4 as inner walls.

In row 5, EchoNeRF “sees” the reflections from the outer walls, and hence the results in row 5 can bring the outer walls as well.

We hope this helps to clarify the situation.

Q4. The authors discuss NeWRF, but don't offer quantitative comparison. Fig 1 is helpful and clearly shows this method isn't adequate for reconstruction (so this point is fairly minor), but it would be helpful to at least show qual. results on one of the same scenes as EchoNeRF for apples to apples comparison.

We understand that apples-to-apples comparison with NeWRF would be helpful, hence we have now run the following experiment for the 4th floorplan in Figure 5. We placed 500 receivers (Rx), spread across the floorplan, and one transmitter (Tx) located at the center. Results from training NeWRF using their official codebase do not show any floorplan. Instead, the learnt geometry has somewhat similar behavior as that of Fig 2(c), where black dots are scattered inside and outside the actual floorplan at virtual Tx locations, but not aligned with walls of the house.

This is not surprising. As authors of the NeWRF paper themselves acknowledged, their neural model fits the signal measurements well but does not solve the core inverse problem to learn the physics of signal propagation (which is in turn important to predict the floorplan).

We will include this qualitative comparison in our revised manuscript.

Q5. It would be helpful if the Fig 5 caption was a bit more descriptive (e.g. mention how the reader can interpret the red stars and lightly colored lines in the first row

Of course. We realized this as we responded to Q3 above.

Q6. L154 The nomenclature for $\omega_j$ is confusing - if $\omega_j$ is an angle, why is the set of discrete values = (1, 2, 3, …) instead of (45, 90, 135, …)? It did not become clear that these values should be multiplied by $45^\circ$ until later in the text.

Yes, we were sloppy here. We will make this precise.

Q7. Why is voxel orientation used as an output of the NIR rather than using more traditional neural SDF approaches that represent scene geometry and can easily be used to compute reflected ray directions?

During the optimization, one can indeed calculate the voxel’s orientation using the voxel opacities. However we found empirically (and as also observed in several papers [1,2]), the orientations inferred from opacities, or via SDF, are noisy during the learning process. These intermediate noisy orientations destabilizes EchoNeRF’s ability to learn the reflections. This is the motivation to explicitly model the orientations per voxel.

[1] Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

[2] Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields

Q8. A brief background section on RF in text would be very helpful for a broad audience like NeurIPS. This could briefly describe at a high level questions, such as how do we interpret the data? Is there a temporal dimension or is it just an intensity image?

Thanks for this suggestion; it makes sense to add a brief review for the wider audience. We will add a background section in the revised version.

Q9. In training stage 1, is there any way to filter out pixels that correspond to occlusions and only supervise other pixels?

We understand your intuition, however, please note an occluding pixel can be located anywhere along the line joining the Tx and the Rx. For all these possible locations, the received LoS power will be the same, hence the LoS model (in stage 1) does not have a way to pin down the correct occluding pixel. To address this, the second stage training is helping to correctly position the occluding pixel, by optimizing for the occlusion pixel that best explains the reflections. This is why filtering out the occluding pixels is not suitable. This motivates the need for the 2-stage training.

Thanks for the clarification. Although the paper is missing real-world evaluation, the rebuttal mitigated my main concerns. Based on the rebuttal, I would like to increase my rating to borderline accept. I encourage the authors to reword L22 to say "vanilla NeRF" rather than "optical NeRF" to be more clear, add the missing related works, and add the new ablation on noise to the supplementary materials in the next revision.

Thank you for your careful consideration and the thoughtful questions. We are grateful.

Our SNR results are actually very conservative (i.e., worst case analysis) so results reported in the rebuttal Table has robustness built in, to cope with noisy real-world conditions. Here is why.

Real-world WiFi SNR is typically between 30dB to 60dB depending on the distance of the device from the WiFi router. It is possible to check SNRs on our own laptops (e.g., on a Macbook, clicking (option + click) on the wifi icon on the top title bar showed us):

RSSI: -46dBm, Noise: -105dBm

This means the laptop’s SNR was 59dB inside a home at a distance of around 8 feet from the router. Another measurement showed 28dB at 51 feet away (farthest location from the home router).

In the rebuttal Table, we have added noise aggressively, so when we report, say, 40dB SNR, it is for the closest receiver that experienced the highest SNR among all receivers. All other receivers experienced lower SNR, and we evaluated EchoNeRF under such weak-SNR scenarios.

In real-world settings, say average SNR is around 45dB. Then the best SNR would be around 60+dB, which corresponds to the first row of our Table. EchoNeRF shows good results for such SNRs as visualized by the floorplans in Figure 5 (and in the Appendix).

We sincerely thank the reviewer for their constructive feedback. Below, we answer some concerns.

Q1. Do the authors have a sense of how this method would work in 3D where scenes exhibit some variation in the added Z dimension?

Yes, we have pondered on this question and see two possible approaches to extend to 3D:

(1) Re-compute the reflection manifolds in 3D, i.e., given the Tx and Rx locations, find which voxels will produce valid reflections. In 2D, these manifolds were curved lines on the 2D plane (as shown in Fig. 3), however, in 3D, the manifolds will become curved lines in the 3D space. Importantly, the number of voxels on these manifolds do not grow excessively, since the 2D manifolds are essentially projections of the 3D manifolds. Hence, we expect the optimization to remain stable.

(2) A more engineering approach could be as follows. When the Tx and Rx are at a similar height $h$, hardly any reflections occur on the vertical walls from higher or lower than $h.$ Hence, we only need to estimate the ceiling and the floor, since the vertical walls can be extended upward and downward from a 2D floorplan. Since ceilings and floors are both horizontal ($90^\circ$ orientation), we can only add a vertical $90^\circ$ manifold in our optimization to model the reflections from the ceiling and floor. Knowing the ceiling and floor heights, and the 2D floorplan, a closed 3D floorplan can be created using existing 3D software.

In follow-up work, we plan to attempt idea #1 first and fall back, if necessary, on idea #2.

Q2. Separately, Have the authors thought about how 2D floorplans may be useful in reconstructing components of indoor scenes with visible light?

Yes, there is indeed a lot of room for innovation in multi-modal fusion. We focussed only on RF in this paper to understand where EchoNeRF alone can get us. However, once the RF mode matures, we see 2 immediate tracks for fusion:

1. Camera helping RF: When cameras can be partially used to view parts of the environment (e.g., kitchen and living room but not the bedroom or bathroom), they can offer valuable initialization to EchoNeRF. This can boost EchoNeRF’s accuracy. Cameras may even be adaptively used to selectively repair parts of the floorplan that exhibit higher uncertainty.
2. RF helping cameras: RF can assist cameras by reducing the number of measurements needed to infer the full scene. This is possible because RF signals can see through walls, which obviates the need for the user to walk to all corners of the house. EchoNeRF can also be used as a second information stream to add robustness to vision-based imaging.

Q3. Is there any reason to expect that parameterizing the occupancy with a signed distance function (SDF) as in [Wang et al. 2021, Yariv et al. 2021, Miller et al. 2024] would improve the quality of the reconstructed floorplans?

Thanks for bringing SDF into our chain of thought. SDF is indeed useful for predicting surface occupancy and may improve the smoothness of walls if we replace opacity with SDFs, as proposed in Wang et al. 2021. However, the wall orientation estimates may be less stable while SDFs are being trained. We found empirically (and as also observed in several papers [1,2]), that the orientations inferred from opacities are noisy during the learning process. That said, we would like to revisit the potential benefits of plugging SDFs into our now-stable EchoNeRF pipeline.

[1] Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

[2] Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields

Q4. Minor weakness: A very minor point, it would have been great to have an example with real world data.

We want to thank you for recognizing that this problem is hard even in simulation. When we started this project, we were envisioning demos of EchoNeRF in the same spirit as camera-based NeRFs. This paper falls short and we acknowledge EchoNeRF has a simulation-to-real gap. However, the core inverse problem with RF proved to be hard (in our opinion) even in a simulation setting; we realized the hardness as we iterated through many failures before finally solving it.

Our next step is to add the necessary pieces to EchoNeRF so it can function in real scenarios. These pieces include, but are not limited to, fusing with RGB, audio, LIDAR or other sensors; using channel impulse responses (CIR) to extract more information from each measurement; leveraging multiple frequencies in WiFi/6G signals; using floorplan priors; and even post-processing on the final results. These should help in narrowing the sim2real gap.

We sincerely thank the reviewer for their constructive feedback. Below, we answer some concerns.

Q1. One major weakness in the presented manuscript is that the evaluation is limited to the ZinD dataset [9], which primarily contains sparsely furnished and geometrically simple rectangular rooms. Expanding the experiments to include more complex datasets such as MVL-dataset or HM3D may enhance the discussion of the limitations of the proposed EchoNeRF.

Thank you for pointing to those datasets. Yes, greater floorplan complexity (complex wall patterns, furniture, clutter) will all help to bring out the deficiencies in EchoNeRF. The goal with this work was not so much to understand the limits of RF based NeRFs, but to establish the feasibility of such frameworks. Given that past work could hardly infer any geometry, our goal was to explore if NeRFs can be taught RF reflections, and if that training can converge to a reasonable floorplan. Frankly, and perhaps naively, we have been surprised at how much NeRFs could infer even when the measurements are just signal power (not channel impulse responses (CIR)). We believe there are many untapped opportunities that can be pursued in follow-up work to close the sim2real gap. Examples of such opportunities are: using priors, fusion with other modalities such as camera or audio, utilizing multiple RF frequencies, using CIR measurements, etc.

Q2. Did the authors consider to evaluate EchoNeRF in environments with walls exhibiting different RF absorption and reflection properties. For example, brick, wood, concrete, and glass windows? If so, how do the reconstruction results differ across such diverse environments?

We ran some experiments this week to observe the sensitivity to materials: we tested for 5 materials across the 6 floorplans shown in Fig 5. We report the mean IoU for EchoNeRF_LoS and EchoNeRF below:

Materials with higher reflectivity, such as concrete and glass, yield better performance than absorptive materials like wood. This is because more reflections allow better performance for EchoNeRF’s reflection model.

Q3. Since multipath signals arise not only from walls but also from ceilings and floors, can the proposed EchoNeRF reconstruct full 3D room geometry, including floor and ceiling. If so, I suggest you to add those experiments to enhance your novelty and the discussion for future works.

Thanks. We have pondered on this question and see two possible approaches to extend to 3D:

(1) Re-compute the reflection manifolds in 3D, i.e., given the Tx and Rx locations, find which voxels will produce valid reflections. In 2D, these manifolds were curved lines on the 2D plane (as shown in Fig. 3), however, in 3D, the manifolds will become curved lines in the 3D space. Importantly, the number of voxels on these manifolds do not grow excessively, since the 2D manifolds are essentially projections of the 3D manifolds. Hence, we expect the optimization to remain stable.

(2) A more engineering approach could be as follows. When the Tx and Rx are at a similar height $h$, hardly any reflections occur on the vertical walls from higher or lower than $h.$ Hence, we only need to estimate the ceiling and the floor, since the vertical walls can be extended upward and downward from a 2D floorplan. Since ceilings and floors are both horizontal ($90^\circ$ orientation), we can only add a vertical $90^\circ$ manifold in our optimization to model the reflections from the ceiling and floor. Knowing the ceiling and floor heights, and the 2D floorplan, a closed 3D floorplan can be created using existing 3D software.

We plan to add this discussion to the future work section of the revised version. In follow-up work, we plan to attempt idea #1 first and fall back, if necessary, on idea #2.

Q4. Fig. 3 requires significant improvement. In its current form, it is really difficult to interpret and does not effectively convey the intended information. For instance, the purple shadow (representing the manifold of reflexions at=45) present incident and reflected lines that is not at in the illustration. Can you elaborate on this?

We now see how this figure can be confusing; we will redraw it in the revised version. Let us try to explain briefly what the purple curve, labeled $45^\circ$, means.

Say a voxel has an orientation of $45^\circ$ with respect to a horizontal X axis. For a given Tx and Rx, at what location should we place this voxel so that the signal from the Tx would reflect on this voxel and reach the Rx? Observe that only certain locations will satisfy this requirement. The locus of all these locations is the purple line.

Q5. In L154-L157, the paper stands that $\omega_j$ = [1, … K] with $K_\omega$ = 4.. However, in L158 and Fig.3, $\omega_j =45^{\circ}$. can you elaborate on this? Is this related to the number of reflations of the signal and the shape of the room? if so, how this affect more complex room shapes? What happens if $K_\omega$ is overestimated, e.g 64? what about underestimated e.g. 1?

That was a typo and thank you for catching it. $K_\omega$ denotes the number of possible discrete orientations for wall voxels, where orientation $\omega$ is in $[0, 180)$. When $K_\omega = 4$, we can write $\omega_j = j\frac{180}{K_\omega}$.

In such settings, walls that are oriented at $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$ will be searched for by the EchoNeRF model. Higher values of $K_\omega$ will benefit the estimation of floorplans with more complex wall structures, while lower $K_\omega$ will need more investigation, an item for our future work.

Q6. I recommend that the authors expand the discussion in Section 5 to address the challenges associated with multiple receivers (RX) and transmitters (TX). open challenges in complementing previous solution that uses RGB cameras or depth/lidar sensors (multimodal approaches), and open concerns related to data privacy.

Thanks for the thoughtful suggestion. We were unable to discuss several of these open questions due to the lack of space in the submission draft. If accepted, we will create space to discuss the open challenges to help facilitate follow-up research after EchoNeRF.