Fiber Monte Carlo

Thank you for taking a close read of our work and dedicating the time to review it. We agree that a comprehensive and deep investigation of just one or two applications is often the right approach. We felt some tension around this ourselves, but in this work, our aim was to establish the generality of our method, as applied to several (seemingly) disparate areas of application.

The tradeoff in doing so is that each of these application areas are research areas in their own right of course (which we can’t do justice to within a few pages of writing). That is all to say, the core (application-oriented) claim we are making is that our method solves a novel problem with a surprisingly far reach in applications. Further work could, for example, exhaustively analyze fiber Monte Carlo applied to topology optimization, but this work is about explicating the fundamental method. Note that we evaluate topology optimization on two examples, not one (though again, we of course could have evaluated three, or seven).

We considered differentiable rendering more broadly to be out of scope of the paper, exactly in the spirit written above (although we are fans of the work you listed). We chose to consider image stylization specifically because it is an appropriate target for fiber Monte Carlo, and no general method had been used to achieve stylization in the literature. Further, it’s worth noting that image stylization is an application which lacks a great/insightful quantitative metric. Technically one frames the application as an optimization problem, but this isn’t the complete picture since solving the problem as posed would not actually result in a stylized image (the minimum of the objective is attained by rendering the original target image perfectly, which hardly “stylizes” it). Here we felt the relevant comparison to our method is attempting to stylize the image without using a procedure to estimate derivatives with respect to geometry (instead only with respect to color), and we provide the results attained by doing so in a side-by-side comparison (see figure 2).

Finally, our application of approximately computing convex hulls is not an application that has been seriously examined in the literature, to our knowledge. We display out-of-sample examples from our data (figure 4) and report out-of-sample accuracy on the intersection over union objective. We could devise yet another method to approach the problem, and then attempt to compare two novel methods, but we don’t feel that would helpfully contextualize the results given this problem has not been approached previously in the literature to our knowledge.

You ask a great question about non-uniform sampling. We have not explicitly attempted to work out the details of the scenario you mention. Speculatively, if one can push the fibers through the inverse function (possibly representing the fibers as parametric curves), it should still be possible to apply our implicit function formulation to estimate the integrals. This is an interesting use-case that should probably be the focus of a follow-up study. Happy to hear your thoughts or speculations on this as well!

Dear reviewer, great to hear from you again! Thanks for taking the time to review (again) our work. We’ll be brief in our response to your two questions.

1. As you mentioned, one previous reviewer was concerned about the level of precision offered in our technical arguments. In this revision, we treat an argument for correctness using measure theory directly and cite relevant/similar results from integral geometry. We felt concerned that this argument might not be as generous/clear as our previous one, so we also cleaned-up and simplified an argument for correctness using just calculus, which is in the appendix. The writing overall was picked through and re-written throughout to improve clarity, and we feel the paper reads much better this time around.
2. Absolutely, we will include this plot: thanks for this suggestion.

Thank you for your helpful feedback. We sincerely appreciate the time and effort you’ve taken to read the work closely and provide valuable feedback. As we understand it, you’re chiefly concerned about the experimental evaluations. We’ll try to provide some context around the experiments and address your specific questions.

At a broad level, we feel that the contribution of this work is a novel method for solving problems which were not previously generically solvable. We do not take the position that our performance on any of the applications surveyed is superior to state of the art. Our position is that performance is clearly comparable, even without domain-specific adaptations to the core method.

All that said, we address your specific concerns in the following paragraphs.

Image Stylization

You write that “it seems clear that the image stylization is meant to stand in for inverse graphics more generally”. As follows from the paragraph above: we do not mean to imply or claim that the image stylization application we present is meant to “stand in for inverse graphics more generally.” We chose image stylization in particular (as a specific sub-problem within inverse graphics) because the objective function contains an integral with a parametric discontinuity; this makes it an appropriate target for fiber Monte Carlo.

Topology Optimization

It is true that the geometry representation is still a grid. In general, level-set topology optimization is known to be very difficult to do stably. Using flexible representations, although in theory possible, is difficult to get right. Existing literature has focused on grid representations, so to align with existing work we chose to stick with the grid representation. Also of note is that topology optimization is a large field with a vast literature, and we do not expect to show that our method is the “best” out of the box on two examples; the marginal improvement in objective function is possibly only incidental. A rigorous evaluation of fiber sampling with topology optimization potentially deserves its own paper; here we simply present that it performs reasonably out of the box compared to a standard approach from the literature, since the focus of this work is about explicating the fundamental method of fiber sampling.

Approximate Convex Hulls

Visually, we see that in certain portions of the image it is not immediately apparent that the approximate hulls are convex sets (the pale coloring makes it hard to distinguish); this is useful feedback for regenerating finalized figures with a better color set. Rest assured that this is a plotting artifact: the approximate hulls are convex by construction. We take the intersection of a finite set of half-spaces. Since a halfspace is a convex set, and the intersection of a finite set of convex sets is itself convex, the approximate hulls are convex (for any value of the parameter).

It is true that our approximate hulls are not supersets of the true convex hulls (i.e., they do not contain all the points, as you mention). This why we use the term “approximate” convex hulls (necessarily convex sets which are not necessarily equal to the convex hull). We do not propose the replacement of classical convex hull algorithms, and completely agree that such a proposal would require significantly more comprehensive analysis (in fact, this is an immense area of study and could be the subject of an entire paper).

Image stylization: I accept that this is an application in its own right, and it is not intended to stand in for inverse graphics more generally.

Approximate convex hulls: The authors say that the approximate convex hulls are formed as the intersections of half-spaces, which would indeed make them convex by construction. However, the fact that the hypernetwork predicts intersections of half-spaces is only alluded to in both the paper and the supplementary material. It would be good to make this explicit. The text only specifies that "[t]he forward model $h : \mathbb{R}^m \times \mathbb{R}^{n\times d} \to \mathbb{R}^p$ is a ‘hypernetwork’ in the sense that it produces a $p$-vector of parameters associated with an implicit function." Later it is mentioned that half-spaces are sampled, but it is unclear from the text how they are used.

Moreover, it is still unclear to me where an approximate convex hull algorithm can be useful if it is not guaranteed to be conservative. E.g., in collision detection, false positives are OK as they can later be resolved, but false negatives are not.

Topology Optimization: This seems like the most compelling application, and it would be great to expand on this a bit more, perhaps by cutting or reworking the approximate convex hull section.