Understanding Continuous-depth Networks through the Lens of Homogeneous Ricci Flows

Despite the claims of contribution, it is not clear that what is the main technical/theoretical results this paper has provided. This is mainly caused by the presentation and organization. It seems that no theoretical guarantee has been shown in any place in the main article.

It could be an interesting direction that Ricci flow and discrete depth networks come to a common ground, and I believe this is not well understood by the machine learning community. However, the layout of section 3 is far from being able to attract general audience of this venue.

The paper is really hard to follow and the authors don’t do a great job at explaining the (admittedly complex) concepts needed in this work. What made it however even tougher for me to follow is that it is very difficult to understand when and where exactly the structure of the neural network is actually used in the theory. The authors start with a rather abstract introduction to Ricci flow, Ricci solitons etc, which is great to have, but then the connection to continuous-depth networks happens very suddenly and in an unclear manner. What exactly is the structure of the neural network determining in the equations? Or differently put, which quantities previously kept abstract (e.g. the manifold, the metric tensor at time $t$, the Ricci curvature etc) is now determined by assuming a neural network structure? I believe it is the diffeomorphisms?

Also, why are we working with the homogeneous Ricci flow instead of the standard Ricci flow? The authors also use a lot of concepts without introducing them, what is ad(K)-invariant for instance? Such things might be obvious to researchers closely working in this field, but even for people interested in theoretical ML research, this paper is really tough to read. There are lots of statements like “There is no doubt that after the discretisation, Eq 11 will degenerate into Eq 4” that are not obvious to me at least.

I think the ideas in this work are interesting, but the work in its current shape really does not explain them well, making it very difficult for me to assess this work positively. I’m happy to re-consider my score if the authors can clarify and potentially incorporate my feedback.

• The naming of "Ricci Flow" is incorrect/misleading. Really, this is an intrinsic geometric flow, of which the Ricci Flow is a special case. This is because the intrinsic geometric flow evolves the metric according to some diff eq (in this case one parameterized by the neural network), whereas the Ricci Flow is a prescribed partial differential equation that doesn't depend on the neural network.
• The method can be simplified considerably in presentation (effectively removing most of the unnecessary manifold/homogenous space constructions). In particular, the real question is how does the jacobian $J$ evolve according to time (the other stuff is used to make sure it doesn't degenerate), as the pullback metric is just the inverse of $J^T J$ (from which one can compute the evolution $\frac{d J^T J}{dt} = J^T \frac{dJ}{dt} + \frac{dJ}{dt}^T J$), which is already well known. This is actually cleaner than the current method, which doesn't utilize the fact that $\frac{dJ}{dt}$ is known for ODEs and instead has to approximate with a step size.
• The method has an intrinsic limitation since the Jacobian scales poorly with input/output dimension.
• Experimentally, the results are only shown for extremely toy 2d data through visualization. Beyond showing ``the method can extract something", this section doesn't convey much else.